Python - 快速批量修改PNG [英] Python - Quick batch modification of PNGs
问题描述
我写了一个python脚本,以独特的方式为OpenGL着色器组合图像。问题是我有大量非常大的地图,需要很长时间才能处理。有没有办法以更快的方式写这个?
I wrote a python script with combines images in unique ways for an OpenGL shader. The problem is that I have a large number of very large maps and it takes a long time to process. Is there a way to write this in a quicker fashion?
import numpy as np
map_data = {}
image_data = {}
for map_postfix in names:
file_name = inputRoot + '-' + map_postfix + resolution + '.png'
print 'Loading ' + file_name
image_data[map_postfix] = Image.open(file_name, 'r')
map_data[map_postfix] = image_data[map_postfix].load()
color = mapData['ColorOnly']
ambient = mapData['AmbientLight']
shine = mapData['Shininess']
width = imageData['ColorOnly'].size[0]
height = imageData['ColorOnly'].size[1]
arr = np.zeros((height, width, 4), dtype=int)
for i in range(width):
for j in range(height):
ambient_mod = ambient[i,j][0] / 255.0
arr[j, i, :] = [color[i,j][0] * ambient_mod , color[i,j][1] * ambient_mod , color[i,j][2] * ambient_mod , shine[i,j][0]]
print 'Converting Color Map to image'
return Image.fromarray(arr.astype(np.uint8))
这只是大量批处理过程的一个示例,所以如果有更快的方法,我会更感兴趣迭代并修改图像文件。几乎所有的时间都用在嵌套循环上,而不是加载和保存。
This is just a sample of a large number of batch processes so I am more interested in if there is a faster way to iterate and modify an image file. Almost all the time is being spent on the nested loop vs loading and saving.
推荐答案
矢量化代码示例 - 在 timeit
或 zmq.Stopwatch()
Vectorised-code example -- test effect on yours in timeit
or zmq.Stopwatch()
报告有22.14秒>> 0.1624秒加速!
Reported to have 22.14 seconds >> 0.1624 seconds speedup!
虽然你的代码似乎只是在RGBA [x,y]上循环,让我展示一个 vectorised - 代码的语法,受益于 numpy
矩阵操作实用程序(忘记RGB / YUV操作(最初基于OpenCV而不是PIL),但重新使用矢量化语法方法以避免for循环和使其适应你的微积分。错误的操作顺序可能会使你的处理时间增加一倍以上。
While your code seems to loop just over RGBA[x,y], let me show a "vectorised"-syntax of a code, that benefits from numpy
matrix-manipulation utilities ( forget the RGB/YUV manipulation ( originally based on OpenCV rather than PIL ), but re-use the vectorised-syntax approach to avoid for-loops and adapt it to work efficiently for your calculus. Wrong order of operations may more than double yours processing time.
并使用测试/优化/重新测试循环来加速。
And use a test / optimise / re-test loop for speeding up.
测试时,使用标准python timeit
如果 [毫秒]
分辨率足够。
For testing, use standard python timeit
if [msec]
resolution is enough.
宁愿选择 zmq.StopWatch( )
如果你需要进入 [usec]
分辨率。
Go rather for zmq.StopWatch()
if you need going into [usec]
resolution.
# Vectorised-code example, to see the syntax & principles
# do not mind another order of RGB->BRG layers
# it has been OpenCV traditional convention
# it has no other meaning in this demo of VECTORISED code
def get_YUV_U_Cb_Rec709_BRG_frame( brgFRAME ): # For the Rec. 709 primaries used in gamma-corrected sRGB, fast, VECTORISED MUL/ADD CODE
out = numpy.zeros( brgFRAME.shape[0:2] )
out -= 0.09991 / 255 * brgFRAME[:,:,1] # // Red
out -= 0.33601 / 255 * brgFRAME[:,:,2] # // Green
out += 0.436 / 255 * brgFRAME[:,:,0] # // Blue
return out
# normalise to <0.0 - 1.0> before vectorised MUL/ADD, saves [usec] ...
# on 480x640 [px] faster goes about 2.2 [msec] instead of 5.4 [msec]
在你的情况下,使用 dtype = numpy.int
,猜猜它应该更快到 MUL
首先按环境[:,:,0]
,最后 DIV
进行标准化 arr [:,,:,3] / = 255
In your case, using dtype = numpy.int
, guess it shall be faster to MUL
first by ambient[:,:,0]
and finally DIV
to normalisearr[:,:,:3] /= 255
# test if this goes even faster once saving the vectorised overhead on matrix DIV
arr[:,:,0] = color[:,:,0] * ambient[:,:,0] / 255 # MUL remains INT, shall precede DIV
arr[:,:,1] = color[:,:,1] * ambient[:,:,0] / 255 #
arr[:,:,2] = color[:,:,2] * ambient[:,:,0] / 255 #
arr[:,:,3] = shine[:,:,0] # STO alpha
那么你的算法看起来如何?
人们不必拥有彼得杰克逊令人印象深刻的预算和时间一旦计划,跨越和执行超过3年的巨大数字运算一个新西兰机库,由一群SGI工作站挤满了人,因为他正在制作指环王全数字母版装配线,正是通过逐帧像素操作, 意识到大规模生产管道中的毫秒,微秒甚至纳秒只是重要。
One need not have Peter Jackson's impressive budget and time once planned, spanned and executed immense number-crunching over 3 years in a New Zealand hangar, overcrowded by a herd of SGI workstations, as he was producing "The Lord of The Rings" fully-digital mastering assembly-line, right by the frame-by-frame pixel manipulation, to realise that miliseconds and microseconds and even nanoseconds in the mass-production pipe-line simply do matter.
所以,深呼吸一下测试和重新测试,以便将您的真实图像处理性能优化到您的项目所需的水平。
So, take a deep breath and test and re-test so as to optimise your real-world imagery processing performance to levels that your project needs.
希望这可以帮助您:
# OPTIONAL for performance testing -------------# ||||||||||||||||||||||||||||||||
from zmq import Stopwatch # _MICROSECOND_ timer
# # timer-resolution step ~ 21 nsec
# # Yes, NANOSECOND-s
# OPTIONAL for performance testing -------------# ||||||||||||||||||||||||||||||||
arr = np.zeros( ( height, width, 4 ), dtype = int )
aStopWatch = zmq.Stopwatch() # ||||||||||||||||||||||||||||||||
# /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\# <<< your original code segment
# aStopWatch.start() # |||||||||||||__.start
# for i in range( width ):
# for j in range( height ):
# ambient_mod = ambient[i,j][0] / 255.0
# arr[j, i, :] = [ color[i,j][0] * ambient_mod, \
# color[i,j][1] * ambient_mod, \
# color[i,j][2] * ambient_mod, \
# shine[i,j][0] \
# ]
# usec_for = aStopWatch.stop() # |||||||||||||__.stop
# print 'Converting Color Map to image'
# print ' FOR processing took ', usec_for, ' [usec]'
# /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\# <<< proposed alternative
aStopWatch.start() # |||||||||||||__.start
# reduced numpy broadcasting one dimension less # ref. comments below
arr[:,:, 0] = color[:,:,0] * ambient[:,:,0] # MUL ambient[0] * [{R}]
arr[:,:, 1] = color[:,:,1] * ambient[:,:,0] # MUL ambient[0] * [{G}]
arr[:,:, 2] = color[:,:,2] * ambient[:,:,0] # MUL ambient[0] * [{B}]
arr[:,:,:3] /= 255 # DIV 255 to normalise
arr[:,:, 3] = shine[:,:,0] # STO shine[ 0] in [3]
usec_Vector = aStopWatch.stop() # |||||||||||||__.stop
print 'Converting Color Map to image'
print ' Vectorised processing took ', usec_Vector, ' [usec]'
return Image.fromarray( arr.astype( np.uint8 ) )
这篇关于Python - 快速批量修改PNG的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!