创建线程
Spawning Threads
Overview
Last time we added audio support by taking advantage of SDL's audio functions. SDL started a thread that made callbacks to a function we defined every time it needed audio. Now we're going to do the same sort of thing with the video display. This makes the code more modular and easier to work with - especially when we want to add syncing. So where do we start?
First we notice that our main function is handling an awful lot: it's running through the event loop, reading in packets, and decoding the video. So what we're going to do is split all those apart: we're going to have a thread that will be responsible for decoding the packets; these packets will then be added to the queue and read by the corresponding audio and video threads. The audio thread we have already set up the way we want it; the video thread will be a little more complicated since we have to display the video ourselves. We will add the actual display code to the main loop. But instead of just displaying video every time we loop, we will integrate the video display into the event loop. The idea is to decode the video, save the resulting frame in another queue, then create a custom event (FF_REFRESH_EVENT) that we add to the event system, then when our event loop sees this event, it will display the next frame in the queue. Here's a handy ASCII art illustration of what is going on:
________ audio   _______    _____
|        | pkts |    | |     | to spkr
| DECODE |----->| AUDIO |--->| SDL |-->
|________|    |_______| |_____|
|   video     _______
| pkts |    |
+---------->| VIDEO |
________    |_______| _______
|    |       |    |    |
| EVENT |       +------>| VIDEO | to mon.
| LOOP   |----------------->| DISP. |-->
|_______|<---FF_REFRESH----|_______|
The main purpose of moving controlling the video display via the event loop is that using an SDL_Delay thread, we can control exactly when the next video frame shows up on the screen. When we finally sync the video in the next tutorial, it will be a simple matter to add the code that will schedule the next video refresh so the right picture is being shown on the screen at the right time.
Simplifying Code
We're also going to clean up the code a bit. We have all this audio and video codec information, and we're going to be adding queues and buffers and who knows what else. All this stuff is for one logical unit, viz. the movie. So we're going to make a large struct that will hold all that information called the VideoState.

 typedef struct VideoState {
AVFormatContext *pFormatCtx;
int videoStream, audioStream;
AVStream *audio_st;
PacketQueue audioq;
uint8_t audio_buf[(AVCODEC_MAX_AUDIO_FRAME_SIZE * ) / ];
unsigned int audio_buf_size;
unsigned int audio_buf_index;
AVPacket audio_pkt;
uint8_t *audio_pkt_data;
int audio_pkt_size;
AVStream *video_st;
PacketQueue videoq;
VideoPicture pictq[VIDEO_PICTURE_QUEUE_SIZE];
int pictq_size, pictq_rindex, pictq_windex;
SDL_mutex *pictq_mutex;
SDL_cond *pictq_cond;
SDL_Thread *parse_tid;
SDL_Thread *video_tid;
char filename[];
int quit;
} VideoState;

Here we see a glimpse of what we're going to get to. First we see the basic information - the format context and the indices of the audio and video stream, and the corresponding AVStream objects. Then we can see that we've moved some of those audio buffers into this structure. These (audio_buf, audio_buf_size, etc.) were all for information about audio that was still lying around (or the lack thereof). We've added another queue for the video, and a buffer (which will be used as a queue; we don't need any fancy queueing stuff for this) for the decoded frames (saved as an overlay). The VideoPicture struct is of our own creations (we'll see what's in it when we come to it). We also notice that we've allocated pointers for the two extra threads we will create, and the quit flag and the filename of the movie.
So now we take it all the way back to the main function to see how this changes our program. Let's set up our VideoState struct:

 int main(int argc, char *argv[]) {
SDL_Event event;
VideoState *is;
is = av_mallocz(sizeof(VideoState));

av_mallocz() is a nice function that will allocate memory for us and zero it out.
Then we'll initialize our locks for the display buffer (pictq), because since the event loop calls our display function - the display function, remember, will be pulling pre-decoded frames from pictq. At the same time, our video decoder will be putting information into it - we don't know who will get there first. Hopefully you recognize that this is a classic race condition. So we allocate it now before we start any threads. Let's also copy the filename of our movie into our VideoState.
pstrcpy(is->filename, sizeof(is->filename), argv[1]);
is->pictq_mutex = SDL_CreateMutex();
is->pictq_cond = SDL_CreateCond();
pstrcpy is a function from ffmpeg that does some extra bounds checking beyond strncpy.
Our First Thread
Now let's finally launch our threads and get the real work done:
schedule_refresh(is, 40);
is->parse_tid = SDL_CreateThread(decode_thread, is);
if(!is->parse_tid) {
   av_free(is);
   return -1;
}
schedule_refresh is a function we will define later. What it basically does is tell the system to push a FF_REFRESH_EVENT after the specified number of milliseconds. This will in turn call the video refresh function when we see it in the event queue. But for now, let's look at SDL_CreateThread().
SDL_CreateThread() does just that - it spawns a new thread that has complete access to all the memory of the original process, and starts the thread running on the function we give it. It will also pass that function user-defined data. In this case, we're calling decode_thread() and with our VideoState struct attached. The first half of the function has nothing new; it simply does the work of opening the file and finding the index of the audio and video streams. The only thing we do different is save the format context in our big struct. After we've found our stream indices, we call another function that we will define, stream_component_open(). This is a pretty natural way to split things up, and since we do a lot of similar things to set up the video and audio codec, we reuse some code by making this a function.
The stream_component_open() function is where we will find our codec decoder, set up our audio options, save important information to our big struct, and launch our audio and video threads. This is where we would also insert other options, such as forcing the codec instead of autodetecting it and so forth. Here it is:

 int stream_component_open(VideoState *is, int stream_index) {
AVFormatContext *pFormatCtx = is->pFormatCtx;
AVCodecContext *codecCtx;
AVCodec *codec;
SDL_AudioSpec wanted_spec, spec;
if(stream_index < || stream_index >= pFormatCtx->nb_streams) {
return -;
}
// Get a pointer to the codec context for the video stream
codecCtx = pFormatCtx->streams[stream_index]->codec;
if(codecCtx->codec_type == CODEC_TYPE_AUDIO) {
// Set audio settings from codec info
wanted_spec.freq = codecCtx->sample_rate;
/* .... */
wanted_spec.callback = audio_callback;
wanted_spec.userdata = is;
if(SDL_OpenAudio(&wanted_spec, &spec) < ) {
fprintf(stderr, "SDL_OpenAudio: %s\n", SDL_GetError());
return -;
}
}
codec = avcodec_find_decoder(codecCtx->codec_id);
if(!codec || (avcodec_open(codecCtx, codec) < )) {
fprintf(stderr, "Unsupported codec!\n");
return -;
}
switch(codecCtx->codec_type) {
case CODEC_TYPE_AUDIO:
is->audioStream = stream_index;
is->audio_st = pFormatCtx->streams[stream_index];
is->audio_buf_size = ;
is->audio_buf_index = ;
memset(&is->audio_pkt, , sizeof(is->audio_pkt));
packet_queue_init(&is->audioq);
SDL_PauseAudio();
break;
case CODEC_TYPE_VIDEO:
is->videoStream = stream_index;
is->video_st = pFormatCtx->streams[stream_index];
packet_queue_init(&is->videoq);
is->video_tid = SDL_CreateThread(video_thread, is);
break;
default:
break;
}
}This is pretty much the same as the code we had before, except now it's generalized for audio and video. Notice that instead of aCodecCtx, we've set up our big struct as the userdata for our audio callback. We've also saved the streams themselves as audio_st and video_st. We also have added our video queue and set it up in the same way we set up our audio queue. Most of the point is to launch the video and audio threads. These bits do it:
SDL_PauseAudio();
break;
/* ...... */
is->video_tid = SDL_CreateThread(video_thread, is);
We remember SDL_PauseAudio() from last time, and SDL_CreateThread() is used as in the exact same way as before. We'll get back to our video_thread() function.
Before that, let's go back to the second half of our decode_thread() function. It's basically just a for loop that will read in a packet and put it on the right queue:
for(;;) {
if(is->quit) {
break;
}
// seek stuff goes here
if(is->audioq.size > MAX_AUDIOQ_SIZE ||
is->videoq.size > MAX_VIDEOQ_SIZE) {
SDL_Delay();
continue;
}
if(av_read_frame(is->pFormatCtx, packet) < ) {
if(url_ferror(&pFormatCtx->pb) == ) {
SDL_Delay(); /* no error; wait for user input */
continue;
} else {
break;
}
}
// Is this a packet from the video stream?
if(packet->stream_index == is->videoStream) {
packet_queue_put(&is->videoq, packet);
} else if(packet->stream_index == is->audioStream) {
packet_queue_put(&is->audioq, packet);
} else {
av_free_packet(packet);
}
}

这里没有什么新东西,除了我们给音频和视频队列限定了一个最大值并且我们添加一个检测读错误的函数。格式上下文里面有一个叫做pb的 ByteIOContext类型结构体。这个结构体是用来保存一些低级的文件信息。函数url_ferror用来检测结构体并发现是否有些读取文件错误。
在循环以后,我们的代码是用等待其余的程序结束和提示我们已经结束的。这些代码是有益的,因为它指示出了如何驱动事件--后面我们将显示影像。

 while(!is->quit) {
SDL_Delay();
}
fail:
if(){
SDL_Event event;
event.type = FF_QUIT_EVENT;
event.user.data1 = is;
SDL_PushEvent(&event);
}
return ;

我们使用SDL常量SDL_USEREVENT来从用户事件中得到值。第一个用户事件的值应当是SDL_USEREVENT,下一个是 SDL_USEREVENT+1并且依此类推。在我们的程序中FF_QUIT_EVENT被定义成SDL_USEREVENT+2。如果喜欢,我们也可以 传递用户数据,在这里我们传递的是大结构体的指针。最后我们调用SDL_PushEvent()函数。在我们的事件分支中,我们只是像以前放入 SDL_QUIT_EVENT部分一样。我们将在自己的事件队列中详细讨论,现在只是确保我们正确放入了FF_QUIT_EVENT事件,我们将在后面捕 捉到它并且设置我们的退出标志quit。
得到帧:video_thread
当我们准备好解码器后,我们开始视频线程。这个线程从视频队列中读取包,把它解码成视频帧,然后调用queue_picture函数把处理好的帧放入到图片队列中:

 int video_thread(void *arg) {
VideoState *is = (VideoState *)arg;
AVPacket pkt1, *packet = &pkt1;
int len1, frameFinished;
AVFrame *pFrame;
pFrame = avcodec_alloc_frame();
for(;;) {
if(packet_queue_get(&is->videoq, packet, ) < ) {
// means we quit getting packets
break;
}
// Decode video frame
len1 = avcodec_decode_video(is->video_st->codec, pFrame, &frameFinished,
packet->data, packet->size);
// Did we get a video frame?
if(frameFinished) {
if(queue_picture(is, pFrame) < ) {
break;
}
}
av_free_packet(packet);
}
av_free(pFrame);
return ;
}

在这里的很多函数应该很熟悉吧。我们把avcodec_decode_video函数移到了这里,替换了一些参数,例如:我们把AVStream保存在我 们自己的大结构体中,所以我们可以从那里得到编解码器的信息。我们仅仅是不断的从视频队列中取包一直到有人告诉我们要停止或者出错为止。

把帧队列化
让我们看一下保存解码后的帧pFrame到图像队列中去的函数。因为我们的图像队列是SDL的覆盖的集合(基本上不用让视频显示函数再做计算了),我们需要把帧转换成相应的格式。我们保存到图像队列中的数据是我们自己做的一个结构体。

 typedef struct VideoPicture {
SDL_Overlay *bmp;
int width, height;
int allocated;
} VideoPicture;

我们的大结构体有一个可以保存这些缓冲区。然而,我们需要自己来申请SDL_Overlay(注意:allocated标志会指明我们是否已经做了这个申请的动作与否)。
为了使用这个队列,我们有两个指针--写入指针和读取指针。我们也要保证一定数量的实际数据在缓冲中。要写入到队列中,我们先要等待缓冲清空以便于有位置 来保存我们的VideoPicture。然后我们检查看我们是否已经申请到了一个可以写入覆盖的索引号。如果没有,我们要申请一段空间。我们也要重新申请 缓冲如果窗口的大小已经改变。然而,为了避免被锁定,尽量避免在这里申请(我现在还不太清楚原因;我相信是为了避免在其它线程中调用SDL覆盖函数的原 因)。

 int queue_picture(VideoState *is, AVFrame *pFrame) {
VideoPicture *vp;
int dst_pix_fmt;
AVPicture pict;
SDL_LockMutex(is->pictq_mutex);
while(is->pictq_size >= VIDEO_PICTURE_QUEUE_SIZE &&
!is->quit) {
SDL_CondWait(is->pictq_cond, is->pictq_mutex);
}
SDL_UnlockMutex(is->pictq_mutex);
if(is->quit)
return -;
// windex is set to 0 initially
vp = &is->pictq[is->pictq_windex];
if(!vp->bmp ||
vp->width != is->video_st->codec->width ||
vp->height != is->video_st->codec->height) {
SDL_Event event;
vp->allocated = ;
event.type = FF_ALLOC_EVENT;
event.user.data1 = is;
SDL_PushEvent(&event);
SDL_LockMutex(is->pictq_mutex);
while(!vp->allocated && !is->quit) {
SDL_CondWait(is->pictq_cond, is->pictq_mutex);
}
SDL_UnlockMutex(is->pictq_mutex);
if(is->quit) {
return -;
}
}

这里的事件机制与前面我们想要退出的时候看到的一样。我们已经定义了事件FF_ALLOC_EVENT作为SDL_USEREVENT。我们把事件发到事件队列中然后等待申请内存的函数设置好条件变量。
让我们来看一看如何来修改事件循环:

 for(;;) {
SDL_WaitEvent(&event);
switch(event.type) {
case FF_ALLOC_EVENT:
alloc_picture(event.user.data1);
break;

记住event.user.data1是我们的大结构体。就这么简单。让我们看一下alloc_picture()函数:

 void alloc_picture(void *userdata) {
VideoState *is = (VideoState *)userdata;
VideoPicture *vp;
vp = &is->pictq[is->pictq_windex];
if(vp->bmp) {
// we already have one make another, bigger/smaller
SDL_FreeYUVOverlay(vp->bmp);
}
// Allocate a place to put our YUV image on that screen
vp->bmp = SDL_CreateYUVOverlay(is->video_st->codec->width,
is->video_st->codec->height,
SDL_YV12_OVERLAY,
screen);
vp->width = is->video_st->codec->width;
vp->height = is->video_st->codec->height;
SDL_LockMutex(is->pictq_mutex);
vp->allocated = ;
SDL_CondSignal(is->pictq_cond);
SDL_UnlockMutex(is->pictq_mutex);
}

你可以看到我们把SDL_CreateYUVOverlay函数从主循环中移到了这里。这段代码应该完全可以自我注释。记住我们把高度和宽度保存到VideoPicture结构体中因为我们需要保存我们的视频的大小没有因为某些原因而改变。
好,我们几乎已经全部解决并且可以申请到YUV覆盖和准备好接收图像。让我们回顾一下queue_picture并看一个拷贝帧到覆盖的代码。你应该能认出其中的一部分:

 int queue_picture(VideoState *is, AVFrame *pFrame) {
if(vp->bmp) {
SDL_LockYUVOverlay(vp->bmp);
dst_pix_fmt = PIX_FMT_YUV420P;
pict.data[] = vp->bmp->pixels[];
pict.data[] = vp->bmp->pixels[];
pict.data[] = vp->bmp->pixels[];
pict.linesize[] = vp->bmp->pitches[];
pict.linesize[] = vp->bmp->pitches[];
pict.linesize[] = vp->bmp->pitches[];
// Convert the image into YUV format that SDL uses
img_convert(&pict, dst_pix_fmt,
(AVPicture *)pFrame, is->video_st->codec->pix_fmt,
is->video_st->codec->width, is->video_st->codec->height);
SDL_UnlockYUVOverlay(vp->bmp);
if(++is->pictq_windex == VIDEO_PICTURE_QUEUE_SIZE) {
is->pictq_windex = ;
}
SDL_LockMutex(is->pictq_mutex);
is->pictq_size++;
SDL_UnlockMutex(is->pictq_mutex);
}
return ;
}

这部分代码和前面用到的一样,主要是简单的用我们的帧来填充YUV覆盖。最后一点只是简单的给队列加1。这个队列在写的时候会一直写入到满为止,在读的时 候会一直读空为止。因此所有的都依赖于is->pictq_size值,这要求我们必需要锁定它。这里我们做的是增加写指针(在必要的时候采用轮转 的方式),然后锁定队列并且增加尺寸。现在我们的读者函数将会知道队列中有了更多的信息,当队列满的时候,我们的写入函数也会知道。

FFPLAY的原理(五)的更多相关文章

  1. How Javascript works (Javascript工作原理) (五) 深入理解 WebSockets 和带有 SSE 机制的HTTP/2 以及正确的使用姿势

    个人总结: 1.长连接机制——分清Websocket,http2,SSE: 1)HTTP/2 引进了 Server Push 技术用来让服务器主动向客户端缓存发送数据.然而,它并不允许直接向客户端程序 ...

  2. FFPLAY的原理

    概要 电影文件有很多基本的组成部分.首先,文件本身被称为容器Container,容器的类型决定了信息被存放在文件中的位置.AVI和Quicktime就是容器的例子.接着,你有一组流,例如,你经常有的是 ...

  3. FFPLAY的原理(六)

    显示视频 这就是我们的视频线程.现在我们看过了几乎所有的线程除了一个--记得我们调用schedule_refresh()函数吗?让我们看一下实际中是如何做的: static void schedule ...

  4. FFPLAY的原理(三)

    播放声音 现在我们要来播放声音.SDL也为我们准备了输出声音的方法.函数SDL_OpenAudio()本身就是用来打开声音设备的.它使用一个叫做SDL_AudioSpec结构体作为参数,这个结构体中包 ...

  5. FFPLAY的原理(七)

    同步音频 现在我们已经有了一个比较像样的播放器.所以让我们看一下还有哪些零碎的东西没处理.上次,我们掩饰了一点同步问题,也就是同步音频到视频而不是其它的同 步方式.我们将采用和视频一样的方式:做一个内 ...

  6. FFPLAY的原理(一)

    概要 电影文件有很多基本的组成部分.首先,文件本身被称为容器Container,容器的类型决定了信息被存放在文件中的位置.AVI和Quicktime就 是容器的例子.接着,你有一组流,例如,你经常有的 ...

  7. redis实现cache系统原理(五)

    1. 介绍 cache就是人们所说的缓存.我们这里所说的cache是web上的.对用户来说,衡量一个网站是否具有良好的体验,其中一个标准就是响应速度的快慢.可能网站刚上线,功能还较少,数据库的记录也不 ...

  8. FFPLAY的原理(四)

    意外情况 你们将会注意到我们有一个全局变量quit,我们用它来保证还没有设置程序退出的信号(SDL会自动处理TERM类似的信号).否则,这个线程将不停地运 行直到我们使用kill -9来结束程序.FF ...

  9. FFPLAY的原理(二)

    关于包Packets的注释 从技术上讲一个包可以包含部分或者其它的数据,但是ffmpeg的解释器保证了我们得到的包Packets包含的要么是完整的要么是多种完整的帧. 现在我们需要做的是让SaveFr ...

随机推荐

  1. STL:list用法详解

    list容器介绍 相对于vector容器的连续线性空间,list是一个双向链表,它有一个重要性质:插入操作和删除操作都不会造成原有的list迭代器失效,每次插入或删除一个元素就配置或释放一个元素空间. ...

  2. Cracking the Coding Interview:: 寻找有环链表的环路起始节点

    给定一个有环链表,实现一个算法返回环路的开头节点. 这个问题是由经典面试题-检测链表是否存在环路演变而来.这个问题也是编程之美的判断两个链表是否相交的扩展问题. 首先回顾一下编程之美的问题. 由于如果 ...

  3. JSP连接access数据库

    一个用jsp连接Access数据库的代码. 要正确的使用这段代码,你需要首先在Access数据库里创建一username表,表里面创建两个字符型的字段,字段名分别为:uid,pwd,然后插入几条测试数 ...

  4. 【翻译】在Sencha Touch中创建离线/在线代理

    原文:Creating an Online/Offline proxy in Sencha Touch 概述 在Sencha Touch中,一个常见的需求就是,当设备在没有连接互联网的时候,应用程序必 ...

  5. 【一天一道LeetCode】#10. Regular Expression Matching

    一天一道LeetCode系列 (一)题目 Implement regular expression matching with support for '.' and '*'. '.' Matches ...

  6. MVC学习笔记(一)

    首先感谢慕课网这个平台提供给我的学习机会,感谢PengCheng老师的"MVC架构模式分析与设计课程". 1.数组的声明: $controllerAllow = array('te ...

  7. SpriteBuilder中CCB精灵对象的Sprite frame为什么有时候不能修改

    有时候你会发现CCB中的精灵对象(root节点)的Sprite frame是灰色的,不能修改.因为它是根对象,所以不存在被嵌入其他CCB的情况,那到底是什么原因呢? 可以发现此时的Timeline当前 ...

  8. LeetCode之“树”:Sum Root to Leaf Numbers

    题目链接 题目要求: Given a binary tree containing digits from 0-9 only, each root-to-leaf path could represe ...

  9. 【LaTeX排版】LaTeX使用--入门基础<一>

    经过两个多星期,毕业论文终于写完了.由于自己对Word软件并不是很熟悉,再加上在数模时见识过LaTex的强大之处,于是就决定用LaTex进行论文的排版.使用LaTex可以避免像Word那样换台机器而出 ...

  10. ubuntu下eclipse新建项目没有java project的解决办法

    装好了eclipse之后却发现新建项目没有java project的选项,大致搜索了一下,并没有发现很好的解决方案(大都是让你重新安装什么的),于是开始瞎鼓捣,并且找到了一个方案: 在终端切换到roo ...