如何通过中心裁切和适合宽度/高度来将视频适合动态壁纸? [英] How to fit video in Live wallpaper, by center-crop and by fitting to width/height?

问题描述

我正在制作可以显示视频的动态壁纸.一开始我以为这会很困难，所以有人建议使用OpenGL解决方案或其他非常复杂的解决方案(例如

I'm making a live wallpaper that can show a video. In the beginning I thought this is going to be very hard, so some people suggested using OpenGL solutions or other, very complex solutions (such as this one).

无论如何，为此，我发现了很多地方都在谈论它，并基于此 github库 (其中有一些错误)，我终于使它起作用了.

Anyway, for this, I've found various places talking about it, and based on this github library (which has some bugs), I finally got it to work.

虽然我已经成功地显示了视频，但是与屏幕分辨率相比，我找不到控制视频显示方式的方法.

While I've succeeded showing a video, I can't find the way to control how it's shown compared to the screen resolution.

当前，总是将其拉伸到屏幕尺寸，这意味着(从 此处 ):

Currently it always gets to be stretched to the screen size, meaning that this (video taken from here) :

显示为:

原因是不同的宽高比:560x320(视频分辨率)与1080x1920(设备分辨率).

Reason is the different aspect ratio : 560x320 (video resolution) vs 1080x1920 (device resolution).

注意:我非常了解缩放视频的解决方案，该解决方案可在各种Github存储库中使用(例如 此处 )，但我要询问的是动态壁纸.因此，它没有View，因此它在做事上受到更多限制.更具体地说，解决方案不能具有任何类型的布局，TextureView或SurfaceView或任何其他类型的View.

Note: I'm well aware of solutions of scaling videos, that are available on various Github repositories (such as here), but I'm asking about a live wallpaper. As such, it doesn't have a View, so it's more limited about how to do things. To be more specifically, a solution can't have any kind of layout, a TextureView or a SurfaceView, or any other kind of View.

我尝试使用SurfaceHolder的各个领域和功能，但到目前为止还没有运气.例子:

I tried to play with various fields and functions of the SurfaceHolder, but with no luck so far. Examples:

• setVideoScalingMode -它要么崩溃，要么不执行任何操作.

• setVideoScalingMode - it either crashes or doesn't do anything.

更改 surfaceFrame -相同.

changing surfaceFrame - same.

这是我编写的当前代码(完整项目可用 此处 ):

Here's the current code I've made (full project available here) :

class MovieLiveWallpaperService : WallpaperService() {
    override fun onCreateEngine(): WallpaperService.Engine {
        return VideoLiveWallpaperEngine()
    }

    private enum class PlayerState {
        NONE, PREPARING, READY, PLAYING
    }

    inner class VideoLiveWallpaperEngine : WallpaperService.Engine() {
        private var mp: MediaPlayer? = null
        private var playerState: PlayerState = PlayerState.NONE

        override fun onSurfaceCreated(holder: SurfaceHolder) {
            super.onSurfaceCreated(holder)
            Log.d("AppLog", "onSurfaceCreated")
            mp = MediaPlayer()
            val mySurfaceHolder = MySurfaceHolder(holder)
            mp!!.setDisplay(mySurfaceHolder)
            mp!!.isLooping = true
            mp!!.setVolume(0.0f, 0.0f)
            mp!!.setOnPreparedListener { mp ->
                playerState = PlayerState.READY
                setPlay(true)
            }
            try {
                //mp!!.setDataSource(this@MovieLiveWallpaperService, Uri.parse("http://techslides.com/demos/sample-videos/small.mp4"))
                mp!!.setDataSource(this@MovieLiveWallpaperService, Uri.parse("android.resource://" + packageName + "/" + R.raw.small))
            } catch (e: Exception) {
            }
        }

        override fun onDestroy() {
            super.onDestroy()
            Log.d("AppLog", "onDestroy")
            if (mp == null)
                return
            mp!!.stop()
            mp!!.release()
            playerState = PlayerState.NONE
        }

        private fun setPlay(play: Boolean) {
            if (mp == null)
                return
            if (play == mp!!.isPlaying)
                return
            when {
                !play -> {
                    mp!!.pause()
                    playerState = PlayerState.READY
                }
                mp!!.isPlaying -> return
                playerState == PlayerState.READY -> {
                    Log.d("AppLog", "ready, so starting to play")
                    mp!!.start()
                    playerState = PlayerState.PLAYING
                }
                playerState == PlayerState.NONE -> {
                    Log.d("AppLog", "not ready, so preparing")
                    mp!!.prepareAsync()
                    playerState = PlayerState.PREPARING
                }
            }
        }

        override fun onVisibilityChanged(visible: Boolean) {
            super.onVisibilityChanged(visible)
            Log.d("AppLog", "onVisibilityChanged:" + visible + " " + playerState)
            if (mp == null)
                return
            setPlay(visible)
        }

    }

    class MySurfaceHolder(private val surfaceHolder: SurfaceHolder) : SurfaceHolder {
        override fun addCallback(callback: SurfaceHolder.Callback) = surfaceHolder.addCallback(callback)

        override fun getSurface() = surfaceHolder.surface!!

        override fun getSurfaceFrame() = surfaceHolder.surfaceFrame

        override fun isCreating(): Boolean = surfaceHolder.isCreating

        override fun lockCanvas(): Canvas = surfaceHolder.lockCanvas()

        override fun lockCanvas(dirty: Rect): Canvas = surfaceHolder.lockCanvas(dirty)

        override fun removeCallback(callback: SurfaceHolder.Callback) = surfaceHolder.removeCallback(callback)

        override fun setFixedSize(width: Int, height: Int) = surfaceHolder.setFixedSize(width, height)

        override fun setFormat(format: Int) = surfaceHolder.setFormat(format)

        override fun setKeepScreenOn(screenOn: Boolean) {}

        override fun setSizeFromLayout() = surfaceHolder.setSizeFromLayout()

        override fun setType(type: Int) = surfaceHolder.setType(type)

        override fun unlockCanvasAndPost(canvas: Canvas) = surfaceHolder.unlockCanvasAndPost(canvas)
    }
}

问题

我想知道如何基于ImageView的内容来调整内容的缩放比例，同时又要保持宽高比:

The questions

I'd like to know how to adjust the scale the content based on what we have for ImageView, all while keeping the aspect ratio :

中心裁剪-适合100％的容器(在这种情况下为屏幕)，在需要时在侧面(顶部和底部或左侧和右侧)裁剪.不会拉伸任何东西.这意味着内容似乎不错，但可能不会显示所有内容.
1. fit-center-拉伸以适合宽度/高度
2. 居中放置-设置为原始大小，居中放置，并仅在太大时才拉伸以适合宽度/高度.

1. center-crop - fits to 100% of the container (the screen in this case), cropping on sides (top&bottom or left&right) when needed. Doesn't stretch anything. This means the content seems fine, but not all of it might be shown.
2. fit-center - stretch to fit width/height
3. center-inside - set as original size, centered, and stretch to fit width/height only if too large.

推荐答案

所以我还无法获得您所要求的所有比例类型，但我已经能够使fit-xy和center-crop正常工作使用exo播放器相当容易.完整的代码可以在 https://github.com/yperess/StackOverflow/tree/50091878上看到，当我获得更多信息时，我将对其进行更新.最终，我还将填写MainActivity，以允许您选择缩放类型作为设置(我将通过一个简单的PreferenceActivity进行设置)，并在服务端读取共享的首选项值.

So I wasn't yet able to get all scale types that you've asked but I've been able to get fit-xy and center-crop working fairly easily using exo player. The full code can be seen at https://github.com/yperess/StackOverflow/tree/50091878 and I'll update it as I get more. Eventually I'll also fill the MainActivity to allow you to choose the scaling type as the settings (I'll do this with a simple PreferenceActivity) and read the shared preferences value on the service side.

总体思路是，深入了解MediaCodec已经实现了fit-xy和center-crop，这实际上是访问视图层次结构所需的仅有两种模式.之所以如此，是因为在表面具有重力并按比例缩放以匹配视频大小*最小缩放比例的情况下，适合中心"，适合顶部"，适合底部"实际上只是fit-xy.为了使这些工作正常进行，我认为需要进行的工作是创建一个OpenGL上下文并提供SurfaceTexture.此SurfaceTexture可以用存根Surface包裹，该存根可以传递给exo播放器.加载视频后，我们就可以设置它们的大小，因为我们已经创建了它们.我们在SurfaceTexture上也有一个回调，可以让我们知道框架准备就绪的时间.在这一点上，我们应该能够修改框架(希望只是使用简单的矩阵比例和变换).

The overall idea is that deep down MediaCodec already implements both fit-xy and center-crop which are really the only 2 modes you would need if you had access to a view hierarchy. This is the case because fit-center, fit-top, fit-bottom would all really just be fit-xy where the surface has a gravity and is scaled to match the video size * minimum scaling. To get these working what I believe will need to happen is we'd need to create an OpenGL context and provide a SurfaceTexture. This SurfaceTexture can be wrapped with a stub Surface which can be passed to exo player. Once the video is loaded we can set the size of these since we created them. We also have a callback on SurfaceTexture to let us know when a frame is ready. At this point we should be able to modify the frame (hopefully just using a simple matrix scale and transform).

此处的关键组件是创建exo播放器:

The key components here are creating the exo player:

    private fun initExoMediaPlayer(): SimpleExoPlayer {
        val videoTrackSelectionFactory = AdaptiveTrackSelection.Factory(bandwidthMeter)
        val trackSelector = DefaultTrackSelector(videoTrackSelectionFactory)
        val player = ExoPlayerFactory.newSimpleInstance(this@MovieLiveWallpaperService,
                trackSelector)
        player.playWhenReady = true
        player.repeatMode = Player.REPEAT_MODE_ONE
        player.volume = 0f
        if (mode == Mode.CENTER_CROP) {
            player.videoScalingMode = C.VIDEO_SCALING_MODE_SCALE_TO_FIT_WITH_CROPPING
        } else {
            player.videoScalingMode = C.VIDEO_SCALING_MODE_SCALE_TO_FIT
        }
        if (mode == Mode.FIT_CENTER) {
            player.addVideoListener(this)
        }
        return player
    }

然后加载视频:

    override fun onSurfaceCreated(holder: SurfaceHolder) {
        super.onSurfaceCreated(holder)
        if (mode == Mode.FIT_CENTER) {
            // We need to somehow wrap the surface or set some scale factor on exo player here.
            // Most likely this will require creating a SurfaceTexture and attaching it to an
            // OpenGL context. Then for each frame, writing it to the original surface but with
            // an offset
            exoMediaPlayer.setVideoSurface(holder.surface)
        } else {
            exoMediaPlayer.setVideoSurfaceHolder(holder)
        }

        val videoUri = RawResourceDataSource.buildRawResourceUri(R.raw.small)
        val dataSourceFactory = DataSource.Factory { RawResourceDataSource(context) }
        val mediaSourceFactory = ExtractorMediaSource.Factory(dataSourceFactory)
        exoMediaPlayer.prepare(mediaSourceFactory.createMediaSource(videoUri))
    }

更新:

让它正常工作，我明天需要在发布代码之前对其进行清理，但这是一个预告片...

Got it working, I'll need tomorrow to clean it up before I post the code but here's a sneak preview...

我最终完成的工作基本上是使用GLSurfaceView并将其拆开.如果您查看它的源代码，唯一的缺憾就是无法在墙纸中使用它，因为它仅在连接到窗口时才启动GLThread.因此，如果您复制相同的代码但允许手动启动GLThread，则可以继续.之后，只需缩放到适合的最小比例并移动要绘制的四边形，就可以跟踪屏幕和视频的大小.

What I ended up doing it basically taking GLSurfaceView and ripping it apart. If you look at the source for it the only thing missing that's making it impossible to use in a wallpaper is the fact that it only starts the GLThread when attached to the window. So if you replicate the same code but allow to manually start the GLThread you can go ahead. After that you just need to keep track of how big your screen is vs the video after scaling to the minimum scale that would fit and shift the quad on which you draw to.

代码的已知问题: 1. GLThread有一个小错误，我无法发现.似乎有一个简单的计时问题，当线程暂停时，我接到signallAll()的调用，实际上并没有等待任何东西. 2.我没有费心在渲染器中动态修改模式.应该不会太难.创建引擎时添加首选项侦听器，然后在scale_type更改时更新渲染器.

Known issues with the code: 1. There's a small bug with the GLThread I haven't been able to fish out. Seems like there's a simple timing issue where when the thread pauses I get a call to signallAll() that's not actually waiting on anything. 2. I didn't bother dynamically modifying the mode in the renderer. It shouldn't be too hard. Add a preference listener when creating the Engine then update the renderer when scale_type changes.

更新: 所有问题均已解决. signallAll()之所以抛出，是因为我错过了检查以查看我们是否真正拥有了锁.我还添加了一个侦听器来动态更新比例类型，因此现在所有比例类型都使用GlEngine.

UPDATE: All issues have been resolved. signallAll() was throwing because I missed a check to see that we actually have the lock. I also added a listener to update the scale type dynamically so now all scale types use the GlEngine.

享受！

这篇关于如何通过中心裁切和适合宽度/高度来将视频适合动态壁纸?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！