“没有足够的存储空间来完成此操作”。当base64编码一个zip文件时 [英] "Not enough storage is available to complete this operation" when base64-encoding a zip file
问题描述
以下代码用于将zip文件转换为base64格式。
The below code is for converting a zip file to base64 format.
Dim inByteArray, base64Encoded,
Const TypeBinary = 1
inByteArray = readBytes("F:path/file.zip")
base64Encoded = encodeBase64(inByteArray)
Private Function readBytes(file)
Dim inStream
' ADODB stream object used
Set inStream = CreateObject("ADODB.Stream")
' open with no arguments makes the stream an empty container
inStream.Open
inStream.Type = TypeBinary
inStream.LoadFromFile(file)
readBytes = inStream.Read()
End Function
Private Function encodeBase64(bytes)
Dim DM, EL
Set DM = CreateObject("Microsoft.XMLDOM")
' Create temporary node with Base64 data type
Set EL = DM.CreateElement("tmp")
EL.DataType = "bin.base64"
' Set bytes, get encoded String
EL.NodeTypedValue = bytes
encodeBase64 = EL.Text
End Function
我首先尝试了一个zip文件大小3MB的文件很好。但是,当我尝试使用大小为34 MB的zip文件时,会显示
I tried first with a zip file of size 3MB It worked fine . But when I try with a zip file of size 34 MB it says
没有足够的存储空间来完成此操作!
Not enough storage is available to complete this operation!
encodeBase64 = EL.Text
有什么方法可以处理所有大小的zip文件,因为我的文件大小通常为30MB或
Is there any way that I can handle zip files of all sizes because my file sizes are mostly 30MB or more.
推荐答案
已编辑 2017/01/10-(原始答案保留在底部)
edited 2017/01/10 - (original answer keeped at bottom)
编辑 2017/01/10-(再次)-我的一些超时问题(并非全部)是由磁盘故障引起的。
edited 2017/01/10 - (again) - some (not all) of my problems with timeouts were caused by a disk failure.
输入数据问题通过拆分转换操作来处理。现在,代码已更改为以两种不同的方式处理缓冲:对于小文件(默认情况下,配置为最大$ 10MB
的文件),使用内存流来存储输出,但是对于大文件(大于 10MB
),将使用一个临时文件(请参见代码后的注释)。
Problems with input data were handled by splitting the conversion operations. Now code has been changed to handle buffering in two different ways: for small files (by default configured for files up to 10MB
) a memory stream is used to store the output, but for big files (greater than 10MB
) a temporary file is used (see notes after code).
Option Explicit
Dim buffer
buffer = encodeFileBase64( "file.zip" )
WScript.StdOut.WriteLine( CStr(Len(buffer)) )
Private Function encodeFileBase64( file )
' Declare ADODB used constants
Const adTypeBinary = 1
Const adTypeText = 2
' Declare FSO constants
Const TEMP_FOLDER = 2
' Initialize output
encodeFileBase64 = ""
' Instantiate FileSystemObject
Dim fso
Set fso = WScript.CreateObject("Scripting.FileSystemObject")
' Check input file exists
If Not fso.FileExists( file ) Then
Exit Function
End If
' Determine how we will handle data buffering.
' Use a temporary file for large files
Dim useTemporaryFile
useTemporaryFile = fso.GetFile( file ).Size > 10 * 1048576
' Instantiate the B64 conversion component
Dim b64
Set b64 = WScript.CreateObject("Microsoft.XMLDOM").CreateElement("tmp")
b64.DataType = "bin.base64"
Dim outputBuffer, outputBufferName
If useTemporaryFile Then
' Create a temporary file to be used as a buffer
outputBufferName = fso.BuildPath( _
fso.GetSpecialFolder( TEMP_FOLDER ), _
fso.GetTempName() _
)
Set outputBuffer = fso.CreateTextFile( outputBufferName, True )
Else
' Instantiate a text stream to be used as a buffer to avoid string
' concatenation operations that were generating out of memory problems
Set outputBuffer = WScript.CreateObject("ADODB.Stream")
With outputBuffer
' Two bytes per character, BOM prefixed buffer
.Type = adTypeText
.Charset = "Unicode"
.Open
End With
End If
' Instantiate a binary stream object to read input file
With WScript.CreateObject("ADODB.Stream")
.Open
.Type = adTypeBinary
.LoadFromFile(file)
' Iterate over input file converting the file, converting each readed
' block to base64 and appending the converted text into the output buffer
Dim inputBuffer
Do
inputBuffer = .Read(3145716)
If IsNull( inputBuffer ) Then Exit Do
b64.NodeTypedValue = inputBuffer
If useTemporaryFile Then
Call outputBuffer.Write( b64.Text )
Else
Call outputBuffer.WriteText( b64.Text )
End If
Loop
' Input file has been readed, close its associated stream
Call .Close()
End With
' It is time to retrieve the contents of the text output buffer into a
' string.
If useTemporaryFile Then
' Close output file
Call outputBuffer.Close()
' Read all the data from the buffer file
encodeFileBase64 = fso.OpenTextFile( outputBufferName ).ReadAll()
' Remove temporary file
Call fso.DeleteFile( outputBufferName )
Else
' So, as we already have a Unicode string inside the stream, we will
' convert it into binary and directly retrieve the data with the .Read()
' method.
With outputBuffer
' Type conversion is only possible while at the start of the stream
.Position = 0
' Change stream type from text to binary
.Type = adTypeBinary
' Skip BOM
.Position = 2
' Retrieve buffered data
encodeFileBase64 = CStr(.Read())
' Ensure we clear the stream contents
.Position = 0
Call .SetEOS()
' All done, close the stream
Call .Close()
End With
End If
End Function
内存是否会成为问题?
是。可用内存仍然是一个限制。无论如何,我已经使用 cscript.exe
作为32位进程运行了90MB文件,并在64bit模式下运行了500MB文件而没有问题。
Yes. Available memory is still a limit. Anyway I have tested the code with cscript.exe
running as a 32bit process with 90MB files and in 64bit mode with 500MB files without problems.
为什么要使用两种方法?
-
流
方法更快(所有操作都在内存中完成,没有字符串连接),但是它需要更多内存,因为在函数末尾它将具有相同数据的两个副本:内部将有一个副本流和将返回的字符串中的一个
The
stream
method is faster (all operations are done in memory without string concatenations), but it requires more memory as it will have two copies of the same data at the end of the function: there will be one copy inside the stream and one in the string that will be returned
临时文件方法比较慢,因为缓冲的数据将被写入磁盘,但由于数据的一个副本,它需要较少的内存。
The temporary file method is slower as the buffered data will be written to disk, but as there is only one copy of the data, it requires less memory.
10MB $ c用于确定是否使用临时文件的$ c>限制只是为了防止32位模式下出现问题的 pesimistic 配置。我已经在32位模式下处理了90MB的文件,没有问题,但是为了确保安全。
The 10MB
limit used to determine if we will use or not a temporary file is just a pesimistic configuration to prevent problems in 32bit mode. I have processed 90MB files in 32bit mode without problems, but just to be safe.
为什么流
配置为 Unicode
,并通过 .Read()
方法检索数据?
Why the stream
is configured as Unicode
and the data is retrieved via .Read()
method?
因为 stream.ReadText()
慢。在内部,它会进行很多字符串转换/检查(是的,建议在文档)使它在这种情况下不可用。
Because the stream.ReadText()
is slow. Internally it makes a lot of string conversions/checks (yes, it is advised in the documentation) that make it unusable in this case.
下面是原始答案。它更简单并避免了转换中的内存问题,但是对于大文件来说,这是不够的。
Below it is the original answer. It is simpler and avoids the memory problem in the conversion but, for large files, it is not enough.
读/编码过程
Option Explicit
Const TypeBinary = 1
Dim buffer
buffer = encodeFileBase64( "file.zip" )
WScript.StdOut.WriteLine( buffer )
Private Function encodeFileBase64( file )
Dim b64
Set b64 = WScript.CreateObject("Microsoft.XMLDOM").CreateElement("tmp")
b64.DataType = "bin.base64"
Dim outputBuffer
Set outputBuffer = WScript.CreateObject("Scripting.Dictionary")
With WScript.CreateObject("ADODB.Stream")
.Open
.Type = TypeBinary
.LoadFromFile(file)
Dim inputBuffer
Do
inputBuffer = .Read(3145716)
If IsNull( inputBuffer ) Then Exit Do
b64.NodeTypedValue = inputBuffer
outputBuffer.Add outputBuffer.Count + 1, b64.Text
Loop
.Close
End With
encodeFileBase64 = Join(outputBuffer.Items(), vbCrLf)
End Function
注意:
-
不,它不是防弹的。您仍然受到构造输出字符串所需空间的限制。对于大文件,您将需要使用输出文件,写入部分结果,直到处理完所有输入为止。
No, it is not bulletproof. You are still limited by the space needed to construct the output string. For big files, you will need to use an output file, writing partial results until all the input has been processed.
3145716仅是54的最接近倍数(每个base64输出行的输入字节数)小于3145728(3MB)。
3145716 is just the nearest multiple of 54 (the number of input bytes for each base64 output line) lower than 3145728 (3MB).
这篇关于“没有足够的存储空间来完成此操作”。当base64编码一个zip文件时的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!