Android Q-从人像模式照片获取深度图 [英] Android Q - Get depth map from portrait mode photos

查看：175
本文介绍了Android Q-从人像模式照片获取深度图的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！

问题描述

我正在尝试构建一个示例Android应用程序，以提取使用Google相机应用程序拍摄的人像模式照片的深度图.我知道它会随模糊的照片一起保存.

I'm trying to build a sample Android App to extract the depth map of portrait mode photos taken with the google camera app. I know that it's saved along the blurred photos.

我阅读了来自Google的动态深度格式文档: https://developer.android.com/training/camera2/Dynamic-depth-v1.0.pdf 这是很新的东西，我找不到与此主题相关的任何资源，也没有找到如何管理android portrait深度图的提取的相关资源.

I read the Dynamic Depth Format documentation coming from Google : https://developer.android.com/training/camera2/Dynamic-depth-v1.0.pdf It's pretty new and I don't find any resource related to this subject or how to manage the extraction of an android portrait's depth map.

我使用元数据提取器库以编程方式读取文件的元数据，尤其是XMP Part，因为它是在其中描述信息的深度的.我在Kotlin中编写了一个示例代码来尝试提取深度图:

I used metadata-extractor library to read file's metadata programmatically, particularly the XMP Part as it's where informations are described for depth. I wrote a sample code in Kotlin to try to extract the depth map:

val inputStream = contentResolver.openInputStream(imageUri)

inputStream?.let { stream ->
    val metadata = JpegMetadataReader.readMetadata(stream)
    val directories = metadata.directories
    val xmpDirectories = metadata.getDirectoriesOfType(XmpDirectory::class.java)

    for (xmpDirectory in xmpDirectories) {
        val xmpMeta = xmpDirectory.xmpMeta
        val itr = xmpMeta.iterator()

        while (itr.hasNext()) {
            val propertyInfo = itr.next() as XMPPropertyInfo

            println(propertyInfo.path + " :: " + propertyInfo.value)
        }
    }

    stream.close()
}

结果输出:

    xmpNote:HasExtendedXMP :: 5c970bbab778024b23c5a8269325455c
    null :: null
    GCreations:CameraBurstID :: 9e286063-a919-4a74-96ee-d7e02d2a17d2
    null :: null
    GCamera:BurstID :: 9e286063-a919-4a74-96ee-d7e02d2a17d2
    GCamera:BurstPrimary :: 1
    GCamera:SpecialTypeID :: 
    GCamera:SpecialTypeID[1] :: com.google.android.apps.camera.gallery.specialtype.SpecialType-PORTRAIT
    null :: null
    Device:Container :: 
    Device:Container/Container:Directory :: 
    Device:Container/Container:Directory[1] :: 
    Device:Container/Container:Directory[1]/Item:Mime :: image/jpeg
    Device:Container/Container:Directory[1]/Item:Length :: 0
    Device:Container/Container:Directory[1]/Item:DataURI :: primary_image
    Device:Container/Container:Directory[1]/rdf:type :: http://ns.google.com/photos/dd/1.0/container/:Item
    Device:Container/Container:Directory[2] :: 
    Device:Container/Container:Directory[2]/Item:Mime :: image/jpeg
    Device:Container/Container:Directory[2]/Item:Length :: 1499039
    Device:Container/Container:Directory[2]/Item:DataURI :: android/original_image
    Device:Container/Container:Directory[2]/rdf:type :: http://ns.google.com/photos/dd/1.0/container/:Item
    Device:Container/Container:Directory[3] :: 
    Device:Container/Container:Directory[3]/Item:Mime :: image/jpeg
    Device:Container/Container:Directory[3]/Item:Length :: 316885
    Device:Container/Container:Directory[3]/Item:DataURI :: android/depthmap
    Device:Container/Container:Directory[3]/rdf:type :: http://ns.google.com/photos/dd/1.0/container/:Item
    Device:Container/Container:Directory[4] :: 
    Device:Container/Container:Directory[4]/Item:Mime :: image/jpeg
    Device:Container/Container:Directory[4]/Item:Length :: 65189
    Device:Container/Container:Directory[4]/Item:DataURI :: android/confidencemap
    Device:Container/Container:Directory[4]/rdf:type :: http://ns.google.com/photos/dd/1.0/container/:Item
    Device:Profiles :: 
    Device:Profiles[1] :: 
    Device:Profiles[1]/Profile:Type :: DepthPhoto
    Device:Profiles[1]/Profile:CameraIndices :: 
    Device:Profiles[1]/Profile:CameraIndices[1] :: 0
    Device:Profiles[1]/rdf:type :: http://ns.google.com/photos/dd/1.0/device/:Profile
    Device:Cameras :: 
    Device:Cameras[1] :: 
    Device:Cameras[1]/Camera:Trait :: Physical
    Device:Cameras[1]/Camera:Image :: 
    Device:Cameras[1]/Camera:Image/Image:ItemSemantic :: Original
    Device:Cameras[1]/Camera:Image/Image:ItemURI :: android/original_image
    Device:Cameras[1]/Camera:DepthMap :: 
    Device:Cameras[1]/Camera:DepthMap/DepthMap:ItemSemantic :: Depth
    Device:Cameras[1]/Camera:DepthMap/DepthMap:Format :: RangeInverse
    Device:Cameras[1]/Camera:DepthMap/DepthMap:Units :: Diopters
    Device:Cameras[1]/Camera:DepthMap/DepthMap:Near :: 0.302570
    Device:Cameras[1]/Camera:DepthMap/DepthMap:Far :: 1.754560
    Device:Cameras[1]/Camera:DepthMap/DepthMap:DepthURI :: android/depthmap
    Device:Cameras[1]/Camera:DepthMap/DepthMap:MeasureType :: OpticalAxis
    Device:Cameras[1]/Camera:DepthMap/DepthMap:ConfidenceURI :: android/confidencemap
    Device:Cameras[1]/Camera:DepthMap/DepthMap:FocalTableEntryCount :: 256
    Device:Cameras[1]/Camera:DepthMap/DepthMap:FocalTable :: heqaPgAAsEGla5s+AACwQZztmz4AALBBbnCcPgAAsEEb9Jw+AACwQah4nT4AALBBFP6dPgAAsEFkhJ4+AACwQZkLnz4AALBBtZOfPsfjrkG7HKA+Xd2sQa2moD7z1qpBjTGhPonQqEFfvaE+H8qmQSNKoj61w6RB3teiPku9okGRZqM+4bagQT/2oz53sJ5B64akPg2qnEGWGKU+o6OaQUSrpT45nZhB+T6mPs+WlkG106Y+ZZCUQX1ppz76iZJBUgCoPpCDkEE4mKg+Jn2OQTIxqT68doxBQsupPlJwikFtZqo+6GmIQbMCqz5+Y4ZBGaCrPhRdhEGiPqw+qlaCQVHerD5AUIBBKX+tPqyTfEEuIa4+2IZ4QWLErj4DenRByWivPi9tcEFnDrA+W2BsQT61sD6HU2hBVF2xPrNGZEGqBrI+3zlgQUSxsj4LLVxBJ12zPjcgWEFVCrQ+YhNUQdS4tD6OBlBBpmi1Prr5S0HPGbY+5uxHQVTMtj4S4ENBOIC3Pj7TP0F/Nbg+asY7QS7suD6WuTdBSaS5PsGsM0HTXbo+7Z8vQdMYuz4ZkytBStW7PkWGJ0E/k7w+cXkjQbZSvT6dbB9BsxO+PslfG0E61r4+9VIXQVOavz4gRhNB/1/APkw5D0FGJ8E+eCwLQSrwwT6kHwdBs7rCPtASA0HjhsM++Av+QMJUxD5P8vVAVCTFPqfY7UCf9cU+/77lQKjIxj5Xpd1AdZ3HPq6L1UALdMg+BnLNQHFMyT5eWMVArCbKPrY+vUDCAss+DSW1QLrgyz5lC61AmcDMPr3xpEBmos0+FdicQCeGzj5svpRA4mvPPsSkjECfU9A+HIuEQGQ90T7n4nhANynSPpevaEAiF9M+RnxYQCgH1D72SEhAU/nUPqUVOECr7dU+VeInQDTk1j4ErxdA+dzXPrR7B0AB2Ng+xpDuP1TV2T4lKs4/9tTaPoTDrT/31ts+41yNP1rb3D6F7Fk/JuLdPkMfGT9q694+AaSwPir33z70Jbw9cAXhPgAAAABEFuI+AAAAALMp4z4AAAAAwz/kPgAAAAB/WOU+AAAAAPJz5j4AAAAAJJLnPgAAAAAgs+g+AAAAAPPW6T4AAAAAo/3qPgAAAAA+J+w+AAAAANBT7T4AAAAAY4PuPgAAAAD/te8+AAAAALfr8D4AAAAAkyTyPgAAAACeYPM+AAAAAOef9D4AAAAAe+L1PgAAAABkKPc+AAAAALJx+D4AAAAAdL75PgAAAAC0Dvs+AAAAAINi/D4AAAAA8Ln9PgAAAAAHFf8+AAAAAOw5AD8AAAAAO+sAPwAAAAB2ngE/AAAAAKZTAj8AAAAA1AoDPwAAAAAIxAM/AAAAAEl/BD8AAAAAojwFPwAAAAAc/AU/AAAAAL69Bj8AAAAAk4EHPwAAAAClRwg/AAAAAP4PCT8AAAAAptoJPwAAAACqpwo/AAAAABN3Cz8AAAAA7EgMPwAAAABAHQ0/AAAAABv0DT8AAAAAic0OPwAAAACTqQ8/AAAAAEmIED8AAAAAtWkRPwAAAADlTRI/AAAAAOQ0Ez8AAACAwx4UPwAAAICNCxU/AAAAgFD7FT8AAACAHO4WPwAAAID/4xc/AAAAgAndGD8AAACASNkZPwAAAIDN2Bo/AAAAgKjbGz8AAACA6+EcPwAAAICm6x0/AAAAgOv4Hj8AAACAzQkgPwAAAIBeHiE/AAAAgLA2Ij8AAACA2lIjPwAAAIDuciQ/AAAAgAGXJT8AAACAKL8mPwAAAIB76yc/AAAAgA8cKT8AAACA/FAqPwAAAIBZiis/AAAAgEDILD8AAACAygouPwAAAIASUi8/AAAAgDCeMD8AAACAQ+8xPwAAAIBmRTM/AAAAgLegND8AAACAUwE2PwAAAIBcZzc/AAAAgO7SOD8AAACALkQ6PwAAAIA+uzs/AAAAgD84PT8AAACAV7s+PwAAAICtREA/AAAAgGTUQT8AAACAqGpDPwAAAICiB0U/AAAAgHurRj8AAACAYVZIPwAAAICACEo/AAAAgAjCSz8AAACALoNNPwAAAIAeTE8/AAAAgA8dUT8AAACAPPZSP3YmPL7W11Q/IaQwvx3CVj9TH5m/SrVYP5Xs2b+bsVo/61wNwFe3XD+Mwy3AucZePy0qTsAM4GA/zpBuwJoDYz+4e4fAqjFlPwivl8CPamc/WeKnwJSuaT+pFbjAEv5rP/pIyMBjWW4/SnzYwN3AcD+br+jA4jRzP+vi+MDYtXU/HosEwSFEeD/GpAzBK+B6P26+FMFpin0/F9gcwaUhgD+/8STBqIWBP2cLLcF58YI/DyU1wV1lhD+4Pj3BmuGFP2BYRcF3Zoc/CHJNwUD0iD+wi1XBSIuKP1mlXcHdK4w/Ab9lwVzWjT+p2G3BG4uPP1HydcF8SpE/+gt+wecUkz/REoPBweqUP6Ufh8F6zJY/eSyLwYq6mD9NOY/BZrWaPyFGk8GUvZw/9lKXwZzTnj/KX5vBCvigP55sn8F6K6M/cnmjwYZupT9GhqfB2sGnPxqTq8ErJqo/7p+vwS6crD/CrLPBsCSvP5e5t8GHwLE/a8a7wY1wtD8/07/BtjW3PxPgw8H4ELo/5+zHwWUDvT+7+cvBHQ7AP48G0MFLMsM/YxPUwThxxj84INjBQ8zJPwwt3MHXRM0/4DngwYnc0D+0RuTB+5TUP4hT6MH5b9g/XGDswXNv3D8wbfDBbZXgPwR69ME
    Device:Cameras[1]/Camera:ImagingModel :: 
    Device:Cameras[1]/Camera:ImagingModel/ImagingModel:FocalLengthX :: 3187.589355
    Device:Cameras[1]/Camera:ImagingModel/ImagingModel:FocalLengthY :: 3187.589355
    Device:Cameras[1]/Camera:ImagingModel/ImagingModel:ImageWidth :: 4032
    Device:Cameras[1]/Camera:ImagingModel/ImagingModel:ImageHeight :: 3024
    Device:Cameras[1]/Camera:ImagingModel/ImagingModel:PrincipalPointX :: 2000.483154
    Device:Cameras[1]/Camera:ImagingModel/ImagingModel:PrincipalPointY :: 1541.417236
    Device:Cameras[1]/Camera:ImagingModel/ImagingModel:Skew :: 0.000000
    Device:Cameras[1]/Camera:ImagingModel/ImagingModel:PixelAspectRatio :: 1.000000
    Device:Cameras[1]/Camera:ImagingModel/ImagingModel:DistortionCount :: 4
    Device:Cameras[1]/Camera:ImagingModel/ImagingModel:Distortion :: AACAPwAAAABBQ7w9AAAAAEgGf74AAAAA5SpFPgAAAAA
    Device:Cameras[1]/rdf:type :: http://ns.google.com/photos/dd/1.0/device/:Camera

根据Google的文档，深度图图像被序列化为base64字符串XMP属性.但我不知道如何提取该图像以基于此深度数据生成新图像.我认为我几乎解决了我的问题，但是我错过了有关Adobe XMP标准的一些认可.

according to the Google's documentation, the depth map image is serialize as a base64 string XMP property. but i don't know how to extract it to generate a new image based this depth data. I think that I almost solved my issue but I miss some acknowledgment about Adobe XMP standard.

我找到了一个叫做"sidecar Xmp文件"的东西，也许我想找到的深度图就在其中.

I found something called "sidecar Xmp files" and maybe the depth map that I'm trying to find is in it.

通过将深度图上传到 https://www.photopea，我设法看到了深度图已嵌入到照片中. com

推荐答案

像素电话肖像模式照片由4个JFIF结构串联而成

Pixel phone portrait mode photo is concatenated of 4 JFIF structure https://en.wikipedia.org/wiki/JPEG_File_Interchange_Format. Each JFIF structure is an jpeg image.

JFIF结构以标记0xFFD8开始，以标记0xFFD9结尾.因此，我们可以将人像模式图像分成4个jpeg文件.

A JFIF structure starts with marker 0xFFD8 and ends with marker 0xFFD9. Therefore, we can split a portrait mode image into 4 jpeg files.

以下python代码打印标记位置，并将PXL_20210107_114027740.PORTRAIT.jpg拆分为

The following python code prints the marker positions and splits PXL_20210107_114027740.PORTRAIT.jpg into,

  1. pxl_out_0.jpg:显示图片
  2. pxl_out_1.jpg:原始图像
  3. pxl_out_2.jpg:带有256级灰度的深度图
  4. pxl_out_3.jpg:填充255个虚拟图像

with open('PXL_20210107_114027740.PORTRAIT.jpg', mode='rb') as infile:
    buffer = infile.read()

bufferlen = len(buffer)
pos = 0
pos_d8 = 0
n = 0
i = 0
while i < bufferlen:
    if buffer[i] == 0xff:
        pos = i
        i += 1
        if buffer[i] == 0xd8:
            print('ffd8: {0}'.format(pos))
            pos_d8 = pos
        elif buffer[i] == 0xd9:
            print('ffd9: {0} len: {1}'.format(pos, pos - pos_d8 + 2))
            with open('pxl_out_{0}.jpg'.format(n), mode='wb') as outfile:
                n += 1
                outfile.write(buffer[pos_d8: pos + 2])
    i += 1

这篇关于Android Q-从人像模式照片获取深度图的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！

查看全文
相关文章
移动开发最新文章
热门教程
热门工具
登录 关闭
扫码关注1秒登录
发送“验证码”获取 | 15天全站免登陆