Blobstore中的图像：获取元数据效率低下？ [英] Images in Blobstore: inefficient to get metadata?

问题描述

摘要：我使用Blobstore让用户上传要传送的图片。我想阻止用户上传无效图像或尺寸过大的文件。我正在使用App Engine的Images服务来获取相关的元数据。但是，为了从Images服务获取有关图像类型或尺寸的任何信息，您必须先执行转换，将转换后的图像提取到App Engine服务器。我已经做了一个无操作的裁剪，并将其编码为非常低质量的JPEG图像，但它仍然获取实际图像，我只想要的是尺寸和文件类型。这是我能做的最好的吗？图像数据（从Blobstore到App Engine服务器）的内部传输会花费我多少钱？

详细信息： b

Blobstore似乎是经过精心设计的，可以有效地提供App Engine中的图像。另一方面，某些操作似乎会让你跳过低效的环节。我希望有人能告诉我，有一种更有效的方式，或者说服我，我所做的并不像我想的那样浪费。

我让用户上传图片作为其他用户生成内容的一部分。 Blobstore使上传和服务非常容易。不幸的是，它允许用户上传他们想要的任何文件，并且我想要施加限制。

（注意：Blobstore的确允许您限制上传的文件大小，但是这个功能的文档记录很差，事实证明，如果用户尝试超出限制，Blobstore将返回一个413实体太大，并且根本不会调用App Engine处理程序。）

我只想要允许有效的JPEG，GIF和PNG文件，并且我想限制尺寸。做这件事的方法似乎是上传后检查文件，如果不允许删除它。这是我得到的：

  class ImageUploadHandler（blobstore_handlers.BlobstoreUploadHandler）：
 def post（self）：
尝试：
＃TODO：检查用户是否已登录并拥有配额; xsrfToken。 
 uploads = self.get_uploads（）
如果len（uploads）！= 1：
 logging.error（'{} files uploaded'.format（len（uploads）））
提高ServerError（'每次上传必须正好是1张图片'）
 image = images.Image（blob_key = uploads [0] .key（））
＃进行无操作转换;否则，execute_transforms（）
＃不起作用，您无法获取任何图像元数据。 
 image.crop（0.0，0.0，1.0，1.0）
 image.execute_transforms（output_encoding = images.JPEG，quality = 1）
 if image.width> 640或image.height> 640：
将ServerError（'图片必须为640x640或更小'）
 resultUrl = images.get_serving_url（uploads [0] .key（））
 self.response.headers ['Content-类型'] ='application / json'
 self.response.body = jsonEncode（{'status'：0，'imageUrl'：resultUrl}）
除了Exception：e：
在上传中：
 blobstore.delete（upload.key（））＃TODO：与delete_async并行删除
 self.response.headers ['Content-Type'] ='text / plain'
 self.response.status = 403 
 self.response.body = e.args [0] 
  

我知道图像可以在服务期间随时调整大小（使用get_serving_url），但我宁愿强制用户首先上传较小的图像，以避免使用存储。后来，我不想限制原始图像尺寸，我可能希望在上传时自动缩小尺寸，但我仍然需要在缩小尺寸和类型之前找出尺寸和类型。

我错过了一种更容易或更有效的方式吗？ 其实Blobstore并没有完全优化为服务图像，它可以处理任何类型的数据。 BlobReader类可用于管理原始Blob数据。

GAE Images服务

a>可用于管理图像（包括在BlobStore中以Blob存储的图像）。你是对的，因为这个服务只提供有关已上传图像的信息在之后执行转换，这对处理之前删除不需要的斑点图像没有帮助。

您可以使用 PIL库中的图像模块 a>（可在 GAE运行时提供的库

PIL图像格式和 size 方法在读取整个图像之前获取您要查找的信息并清理图像数据：

 >>> image = Image.open（'Spain-rail-map.jpg'）
>>> image.format 
'JPEG'
>>> image.size 
（410，317）
  

这些方法应该非常高效，需要由打开方法加载的blob中的图像标题信息：
$ b

打开并识别给定的图像文件。这是一个懒惰的操作;
函数读取文件头，但实际图像数据不是从文件读取的
，直到您尝试处理数据为止（将 load
方法调用为强制加载）。

这是如何在 ImageUploadHandler ：来自PIL的

 将blobstore.BlobReader（上传[0] .key（））映像
作为fd： 
 image = Image.open（fd）
 logging.error（'format =％s'％image.format）
 logging.error（'size =％dx％d'％image。大小）
  

Summary: I'm using Blobstore to let users upload images to be served. I want to prevent users from uploading files that aren't valid images or have dimensions that are too large. I'm using App Engine's Images service to get the relevant metadata. BUT, in order to get any information about the image type or dimensions from the Images service, you have to first execute a transform, which fetches the transformed image to the App Engine server. I have it do a no-op crop and encode as a very low quality JPEG image, but it's still fetching an actual image, and all I want is the dimensions and file type. Is this the best I can do? Will the internal transfer of the image data (from Blobstore to App Engine server) cost me?

Details:

It seems like Blobstore was carefully designed for efficient serving of images from App Engine. On the other hand, certain operations seem to make you jump through inefficient hoops. I'm hoping someone can tell me that there's a more efficient way, or convince me that what I'm doing is not as wasteful as I think it is.

I'm letting users upload images to be served as part of other user-generated content. Blobstore makes the uploading and serving pretty easy. Unfortunately it lets the user upload any file they want, and I want to impose restrictions.

(Side note: Blobstore does let you limit the file size of uploads, but this feature is poorly documented. It turns out that if the user tries to exceed the limit, Blobstore will return a 413 "Entity too large", and the App Engine handler is not called at all.)

I want to allow only valid JPEG, GIF, and PNG files, and I want to limit the dimensions. The way to do this seems to be to check the file after upload, and delete it if it's not allowed. Here's what I've got:

class ImageUploadHandler(blobstore_handlers.BlobstoreUploadHandler):
  def post(self):
    try:
      # TODO: Check that user is logged in and has quota; xsrfToken.
      uploads = self.get_uploads()
      if len(uploads) != 1:
        logging.error('{} files uploaded'.format(len(uploads)))
        raise ServerError('Must be exactly 1 image per upload')
      image = images.Image(blob_key=uploads[0].key())
      # Do a no-op transformation; otherwise execute_transforms()
      # doesn't work and you can't get any image metadata.
      image.crop(0.0, 0.0, 1.0, 1.0)
      image.execute_transforms(output_encoding=images.JPEG, quality=1)
      if image.width > 640 or image.height > 640:
        raise ServerError('Image must be 640x640 or smaller')
      resultUrl = images.get_serving_url(uploads[0].key())
      self.response.headers['Content-Type'] = 'application/json'
      self.response.body = jsonEncode({'status': 0, 'imageUrl': resultUrl})
    except Exception as e:
      for upload in uploads:
        blobstore.delete(upload.key()) # TODO: delete in parallel with delete_async
      self.response.headers['Content-Type'] = 'text/plain'
      self.response.status = 403
      self.response.body = e.args[0]

Comments in the code highlight the issue.

I know the image can be resized on the fly at serve time (using get_serving_url), but I'd rather force users to upload a smaller image in the first place, to avoid using up storage. Later, instead of putting a limit on the original image dimensions, I might want to have it automatically get shrunk at upload time, but I'd still need to find out its dimensions and type before shrinking it.

Am I missing an easier or more efficient way?

解决方案

Actually the Blobstore is not exactly optimized for serving images, it operates on any kind of data. The BlobReader class can be used to manage the raw blob data.

The GAE Images service can be used to manage images (including those stored as blobs in the BlobStore). You are right in the sense that this service only offers info about the uploaded image only after executing a transformation on it, which doesn't help with deleting undesirable blob images prior to processing.

What you can do is use the Image module from the PIL library (available between the GAE's Runtime-Provided Libraries) overlayed on top of the BlobReader class.

The PIL Image format and size methods to get the info you seek and sanitize the image data before reading the entire image:

>>> image = Image.open('Spain-rail-map.jpg')
>>> image.format
'JPEG'
>>> image.size
(410, 317)

These methods should be very efficient since they only need image header info from the blob loaded by the open method:

Opens and identifies the given image file. This is a lazy operation; the function reads the file header, but the actual image data is not read from the file until you try to process the data (call the load method to force loading).

This is how overlaying can be done in your ImageUploadHandler:

  from PIL import Image
  with blobstore.BlobReader(uploads[0].key()) as fd:
      image = Image.open(fd)
      logging.error('format=%s' % image.format)
      logging.error('size=%dx%d' % image.size)

这篇关于Blobstore中的图像：获取元数据效率低下？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！