记录一个Unity播放器插件的开发

效果图##

先上一个效果图：

背景

公司最近在做VR直播平台，VR开发我们用到了Unity，而在Unity中播放视频就需要一款视频插件，我们调研了几个视频插件，记录两个，如下：

Unity视频插件调研

网上搜了搜，最流行的有以下两款Unity插件：

AVPro 这个在Unity商店售价150$，最新release版本为1.6.15,功能包括：

Powerful cross-platform video playback solution for Unity.

Native video playback on Android, iOS, macOS and tvOS (Apple TV), WebGL, Windows, Windows Phone and UWP.

Features include:

New Unity 2017 supported
New New iOS video playback path that uses less memory
One API for video playback on all supported platforms
Unity 4.6 - 5.x supported
8K video (on supported hardware)
VR Support (mono, stereo, equirectangular and cubemap)
Transparency support (native and packed)
Subtitles support (external SRT)
Fast flexible video playback
In-editor playback support for Windows and macOS
Free watermarked trial version available
Components for IMGUI, uGUI and NGUI
Over 64 PlayMaker actions included
Easy to use drag and drop components
Linear and Gamma colour spaces supported
Fast native Direct3D, OpenGL and Metal texture updates
Desktop support for Hap, Hap Alpha, Hap Q and Hap Q Alpha
Streaming video from URL (when supported by platform)

此插件支持HLS视频播放,使用文档很详细，但是此插件没有源码，不适合做以后的个性化开发。

鼎鼎大名的EasyMovieTexture.售价65$，支持功能如下：

Supported resolutions:

Android: General devices support up to 1920 * 1080.

The latest device supports up to 4k.
iOS: General devices support up to 1920 * 1080.

The latest device is support up to 2560 * 1440.

iPhone 6s Plus supports up to 4k.
It also supports StreamingAssets, external storage, and streaming services.
Android streaming support list: http, HLS (http live streaming),rtsp
iOS streaming support list: http,HLS (http live streaming)
EasyMovieTexture requires Android 4.0 or above.
EasyMovieTexture requires iOS 6.0 or Above.
Unity 4.X requires an iOS Pro.
In Unity 5.X it does not require a Pro.
Supports multithreaded rendering options. (Only supports Unity 5.X.)

这个插件貌似是个人开发的，没有说明文档，有部分java源码，native code并没有给出。我们需要有源码的插件方便以后的个性化开发。

自己动手，风衣足食##

综合以上调研结果，我们决定自己动手实现一个简单能满足我们要求的Unity播放器插件，有两个难点要突破：

一个是找一个合适的开源播放器。
另一个就是如何把播放视频画面映射到Unity中的物体表面，这个是最关键的。

寻找素材

从下面这个帖子中，找到了一些可以参考的资料。

unity 3d 中如何实现以物体的表面作为播放视频的位置，比如在墙面播放视频？

寻找开源播放器

本来打算使用VLC播放器的，但是同事发现有一个商用的开源播放器，并且使用的人数也不少，B站的ijkplayer。正好在上面的帖子中回复人也提到了这个播放器，我们决定使用这个播放器。

如何做视频画面映射

没有一点Unity开发经验，只能从头一点点学起，知乎的帖子里面，有个人回复可以参考OVR里面的例子。阅读了里面的代码，同时也参考了easyMovieTexture中的源码（easyMovie中只有java代码，关键的native code并没有给）。看的有些似懂非懂，尝试了之后，居然成功了。

最关键的一点我描述成下面的话：

将Ijkplayer的AndroidSurfaceTexture纹理ID和Unity中Texture2D的纹理ID分别同时绑定到不同的目标上。AndroidSurfaceTexture绑定到GL_TEXTURE_EXTERNAL_OES，Unity的纹理ID绑定到GL_TEXTURE_2D

从头到尾梳理一遍流程

初始化####

Unity

Unity端初始化一个Texture2D纹理ID用于显示视频帧。

m_VideoTexture = new Texture2D (Call_GetVideoWidth (), Call_GetVideoHeight (), TextureFormat.RGB565, false);

这里使用了OVR里面的native code，OVR中初始化AndroidSurfaceTexture和相关的函数：

static const char * className = "android/graphics/SurfaceTexture";

	const jclass surfaceTextureClass = jni->FindClass(className);

	if ( surfaceTextureClass == 0 ) {

		FAIL( "FindClass( %s ) failed", className );

	}

	// find the constructor that takes an int

	const jmethodID constructor = jni->GetMethodID( surfaceTextureClass, "<init>", "(I)V" );

	if ( constructor == 0 ) {

		FAIL( "GetMethodID( <init> ) failed" );

	}

	jobject obj = jni->NewObject( surfaceTextureClass, constructor, textureId );

	if ( obj == 0 ) {

		FAIL( "NewObject() failed" );

	}

	javaObject = jni->NewGlobalRef( obj );

	if ( javaObject == 0 ) {

		FAIL( "NewGlobalRef() failed" );

	}

	// Now that we have a globalRef, we can free the localRef

	jni->DeleteLocalRef( obj );

    updateTexImageMethodId = jni->GetMethodID( surfaceTextureClass, "updateTexImage", "()V" );

    if ( !updateTexImageMethodId ) {

    	FAIL( "couldn't get updateTexImageMethodId" );

    }

    getTimestampMethodId = jni->GetMethodID( surfaceTextureClass, "getTimestamp", "()J" );

    if ( !getTimestampMethodId ) {

    	FAIL( "couldn't get getTimestampMethodId" );

    }

	setDefaultBufferSizeMethodId = jni->GetMethodID( surfaceTextureClass, "setDefaultBufferSize", "(II)V" );

    if ( !setDefaultBufferSizeMethodId ) {

		FAIL( "couldn't get setDefaultBufferSize" );

    }

	// jclass objects are localRefs that need to be freed

	jni->DeleteLocalRef( surfaceTextureClass );

初始化纹理ID，并将其绑定到目标GL_TEXTURE_2D上:

glGenTextures( 1, &textureId );

	glBindTexture( GL_TEXTURE_EXTERNAL_OES, textureId );

	glTexParameterf( GL_TEXTURE_EXTERNAL_OES, GL_TEXTURE_MIN_FILTER, GL_LINEAR );

	glTexParameterf( GL_TEXTURE_EXTERNAL_OES, GL_TEXTURE_MAG_FILTER, GL_LINEAR );

	glTexParameterf( GL_TEXTURE_EXTERNAL_OES, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE );

	glTexParameterf( GL_TEXTURE_EXTERNAL_OES, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE );

	glBindTexture( GL_TEXTURE_EXTERNAL_OES, 0 );

将Unity的纹理ID传递到OVR中，用于绑定到目标GL_TEXTURE_EXTERNAL_OES上：



jobject OVR_Media_Surface( void * texPtr, int const width, int const height )

{

	GLuint texId = (GLuint)(size_t)(texPtr);

	LOG( "OVR_Media_Surface(%i, %i, %i)", texId, width, height );

	return _msp.VideoSurface.Bind( texId, width, height );

}

Ijkplayer

创建一个播放器，注意这里我们使用OVR中已经实例化的AndroidMovieTexture来初始化播放器。

 m_IjkMediaPlayer.setSurface(m_Surface);

刷新####

刷新操作由Unity中的Update函数触发，最终在OVR中执行，首先调用AndroidMovieTexture中的Update函数，接下来就是绑定纹理操作，Ijkplayer的纹理ID每刷新一次绑定一次。而Unity的纹理ID只有在视频图像长度或者宽度发生变化才会绑定。

void MediaSurface::Update()

{

	if ( !AndroidSurfaceTexture )

	{

		LOG( "!AndroidSurfaceTexture" );

		return;

	}

	if ( TexId <= 0 )

	{

		//LOG( "TexId <= 0" );

		return;

	}

	AndroidSurfaceTexture->Update();

	if ( AndroidSurfaceTexture->GetNanoTimeStamp() == LastSurfaceTexNanoTimeStamp )

	{

		//LOG( "No new surface!" );

		return;

	}

	LastSurfaceTexNanoTimeStamp = AndroidSurfaceTexture->GetNanoTimeStamp()

   // If the SurfaceTexture has changed dimensions, we need to

	// reallocate the texture and FBO.

	glActiveTexture( GL_TEXTURE0 );

	glBindTexture( GL_TEXTURE_EXTERNAL_OES, AndroidSurfaceTexture->GetTextureId() );

	if ( TexIdWidth != BoundWidth || TexIdHeight != BoundHeight )

	{

		LOG( "New surface size: %ix%i", BoundWidth, BoundHeight );

		TexIdWidth = BoundWidth;

		TexIdHeight = BoundHeight;

		if ( Fbo )

		{

			glDeleteFramebuffers( 1, &Fbo );

		}

		glActiveTexture( GL_TEXTURE1 );

		glBindTexture( GL_TEXTURE_2D, TexId );

		glTexImage2D( GL_TEXTURE_2D, 0, GL_RGBA,

				TexIdWidth, TexIdHeight, 0, GL_RGBA, GL_UNSIGNED_BYTE, NULL );

		glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE );

		glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE );

		glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR_MIPMAP_LINEAR );

		glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR );

		glBindTexture( GL_TEXTURE_2D, 0 );

		glActiveTexture( GL_TEXTURE0 );

		glGenFramebuffers( 1, &Fbo );

		glBindFramebuffer( GL_FRAMEBUFFER, Fbo );

		glFramebufferTexture2D( GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D,

				TexId, 0 );

		glBindFramebuffer( GL_FRAMEBUFFER, 0 );

	}

}

最后的结果可能是这个样子的：Ijkplayer负责推动视频不停向前播放，播放器的纹理也会不停刷新，这会带动Unity纹理跟着刷新，最终显示在Unity的Material上。