将CGImage转换为python图像(pil/opencv) [英] Converting CGImage to python image (pil/opencv)

问题描述

我想在屏幕上进行一些模式识别，并将使用Quartz/PyObjc库获取屏幕截图.

I want to do some pattern recognition on my screen and will use the Quartz/PyObjc libraries to get the screenshots.

我将屏幕截图作为CGImage获得.我想使用openCV库在其中搜索模式，但似乎找不到如何将数据转换为可由opencv读取的模式.

I get the screenshot as a CGImage. I want to search for a pattern in it using the openCV library, but can't seem to find how to convert the data to be readable by opencv.

所以我要做的是:

#get screenshot and reference pattern
img = getScreenshot() # returns CGImage instance, custom function, using Quartz
reference = cv2.imread('ref/reference_start.png') #get the reference pattern

#search for the pattern using the opencv library
result = cv2.matchTemplate(screen, reference, cv2.TM_CCOEFF_NORMED)

#this is what I need
minVal,maxVal,minLoc,maxLoc = cv2.minMaxLoc(result)

我不知道该怎么做，也无法通过Google查找信息.

I have no idea how to do this and can't find information through google.

推荐答案

要添加到Arqu的答案中，如果您的最终目标是使用opencv或numpy，则可能会发现使用np.frombuffer而不是先创建PIL映像会更快. ，因为np.frombuffer与Image.frombuffer花费的时间大约相同，但省去了从Image转换为numpy数组的步骤(这在我的机器上花费了大约100ms(其他所有花费了〜50ms)).

To add to Arqu's answer, you may find it faster to use np.frombuffer instead of creating a PIL Image first if your ultimate goal is to use opencv or numpy, because np.frombuffer takes about the same time as Image.frombuffer, but saves you the step of converting from an Image to a numpy array (which takes about 100ms on my machine (everything else takes ~50ms)).

import Quartz.CoreGraphics as CG
from PIL import Image 
import time
import numpy as np

ct = time.time()
region = CG.CGRectInfinite

# Create screenshot as CGImage
image = CG.CGWindowListCreateImage(
    region,
    CG.kCGWindowListOptionOnScreenOnly,
    CG.kCGNullWindowID,
    CG.kCGWindowImageDefault)

width = CG.CGImageGetWidth(image)
height = CG.CGImageGetHeight(image)
bytesperrow = CG.CGImageGetBytesPerRow(image)

pixeldata = CG.CGDataProviderCopyData(CG.CGImageGetDataProvider(image))
image = np.frombuffer(pixeldata, dtype=np.uint8)
image = image.reshape((height, bytesperrow//4, 4))
image = image[:,:width,:]

print('elapsed:', time.time() - ct)

这篇关于将CGImage转换为python图像(pil/opencv)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！