FFPLAY的原理(五)
创建线程
Spawning Threads
Overview
Last time we added audio support by taking advantage of SDL's audio functions. SDL started a thread that made callbacks to a function we defined every time it needed audio. Now we're going to do the same sort of thing with the video display. This makes the code more modular and easier to work with - especially when we want to add syncing. So where do we start?
First we notice that our main function is handling an awful lot: it's running through the event loop, reading in packets, and decoding the video. So what we're going to do is split all those apart: we're going to have a thread that will be responsible for decoding the packets; these packets will then be added to the queue and read by the corresponding audio and video threads. The audio thread we have already set up the way we want it; the video thread will be a little more complicated since we have to display the video ourselves. We will add the actual display code to the main loop. But instead of just displaying video every time we loop, we will integrate the video display into the event loop. The idea is to decode the video, save the resulting frame in another queue, then create a custom event (FF_REFRESH_EVENT) that we add to the event system, then when our event loop sees this event, it will display the next frame in the queue. Here's a handy ASCII art illustration of what is going on:
________ audio _______ _____
| | pkts | | | | to spkr
| DECODE |----->| AUDIO |--->| SDL |-->
|________| |_______| |_____|
| video _______
| pkts | |
+---------->| VIDEO |
________ |_______| _______
| | | | |
| EVENT | +------>| VIDEO | to mon.
| LOOP |----------------->| DISP. |-->
|_______|<---FF_REFRESH----|_______|
The main purpose of moving controlling the video display via the event loop is that using an SDL_Delay thread, we can control exactly when the next video frame shows up on the screen. When we finally sync the video in the next tutorial, it will be a simple matter to add the code that will schedule the next video refresh so the right picture is being shown on the screen at the right time.
Simplifying Code
We're also going to clean up the code a bit. We have all this audio and video codec information, and we're going to be adding queues and buffers and who knows what else. All this stuff is for one logical unit, viz. the movie. So we're going to make a large struct that will hold all that information called the VideoState.
typedef struct VideoState {
AVFormatContext *pFormatCtx;
int videoStream, audioStream;
AVStream *audio_st;
PacketQueue audioq;
uint8_t audio_buf[(AVCODEC_MAX_AUDIO_FRAME_SIZE * ) / ];
unsigned int audio_buf_size;
unsigned int audio_buf_index;
AVPacket audio_pkt;
uint8_t *audio_pkt_data;
int audio_pkt_size;
AVStream *video_st;
PacketQueue videoq;
VideoPicture pictq[VIDEO_PICTURE_QUEUE_SIZE];
int pictq_size, pictq_rindex, pictq_windex;
SDL_mutex *pictq_mutex;
SDL_cond *pictq_cond;
SDL_Thread *parse_tid;
SDL_Thread *video_tid;
char filename[];
int quit;
} VideoState;
Here we see a glimpse of what we're going to get to. First we see the basic information - the format context and the indices of the audio and video stream, and the corresponding AVStream objects. Then we can see that we've moved some of those audio buffers into this structure. These (audio_buf, audio_buf_size, etc.) were all for information about audio that was still lying around (or the lack thereof). We've added another queue for the video, and a buffer (which will be used as a queue; we don't need any fancy queueing stuff for this) for the decoded frames (saved as an overlay). The VideoPicture struct is of our own creations (we'll see what's in it when we come to it). We also notice that we've allocated pointers for the two extra threads we will create, and the quit flag and the filename of the movie.
So now we take it all the way back to the main function to see how this changes our program. Let's set up our VideoState struct:
int main(int argc, char *argv[]) {
SDL_Event event;
VideoState *is;
is = av_mallocz(sizeof(VideoState));
av_mallocz() is a nice function that will allocate memory for us and zero it out.
Then we'll initialize our locks for the display buffer (pictq), because since the event loop calls our display function - the display function, remember, will be pulling pre-decoded frames from pictq. At the same time, our video decoder will be putting information into it - we don't know who will get there first. Hopefully you recognize that this is a classic race condition. So we allocate it now before we start any threads. Let's also copy the filename of our movie into our VideoState.
pstrcpy(is->filename, sizeof(is->filename), argv[1]);
is->pictq_mutex = SDL_CreateMutex();
is->pictq_cond = SDL_CreateCond();
pstrcpy is a function from ffmpeg that does some extra bounds checking beyond strncpy.
Our First Thread
Now let's finally launch our threads and get the real work done:
schedule_refresh(is, 40);
is->parse_tid = SDL_CreateThread(decode_thread, is);
if(!is->parse_tid) {
av_free(is);
return -1;
}
schedule_refresh is a function we will define later. What it basically does is tell the system to push a FF_REFRESH_EVENT after the specified number of milliseconds. This will in turn call the video refresh function when we see it in the event queue. But for now, let's look at SDL_CreateThread().
SDL_CreateThread() does just that - it spawns a new thread that has complete access to all the memory of the original process, and starts the thread running on the function we give it. It will also pass that function user-defined data. In this case, we're calling decode_thread() and with our VideoState struct attached. The first half of the function has nothing new; it simply does the work of opening the file and finding the index of the audio and video streams. The only thing we do different is save the format context in our big struct. After we've found our stream indices, we call another function that we will define, stream_component_open(). This is a pretty natural way to split things up, and since we do a lot of similar things to set up the video and audio codec, we reuse some code by making this a function.
The stream_component_open() function is where we will find our codec decoder, set up our audio options, save important information to our big struct, and launch our audio and video threads. This is where we would also insert other options, such as forcing the codec instead of autodetecting it and so forth. Here it is:
int stream_component_open(VideoState *is, int stream_index) {
AVFormatContext *pFormatCtx = is->pFormatCtx;
AVCodecContext *codecCtx;
AVCodec *codec;
SDL_AudioSpec wanted_spec, spec;
if(stream_index < || stream_index >= pFormatCtx->nb_streams) {
return -;
}
// Get a pointer to the codec context for the video stream
codecCtx = pFormatCtx->streams[stream_index]->codec;
if(codecCtx->codec_type == CODEC_TYPE_AUDIO) {
// Set audio settings from codec info
wanted_spec.freq = codecCtx->sample_rate;
/* .... */
wanted_spec.callback = audio_callback;
wanted_spec.userdata = is;
if(SDL_OpenAudio(&wanted_spec, &spec) < ) {
fprintf(stderr, "SDL_OpenAudio: %s\n", SDL_GetError());
return -;
}
}
codec = avcodec_find_decoder(codecCtx->codec_id);
if(!codec || (avcodec_open(codecCtx, codec) < )) {
fprintf(stderr, "Unsupported codec!\n");
return -;
}
switch(codecCtx->codec_type) {
case CODEC_TYPE_AUDIO:
is->audioStream = stream_index;
is->audio_st = pFormatCtx->streams[stream_index];
is->audio_buf_size = ;
is->audio_buf_index = ;
memset(&is->audio_pkt, , sizeof(is->audio_pkt));
packet_queue_init(&is->audioq);
SDL_PauseAudio();
break;
case CODEC_TYPE_VIDEO:
is->videoStream = stream_index;
is->video_st = pFormatCtx->streams[stream_index];
packet_queue_init(&is->videoq);
is->video_tid = SDL_CreateThread(video_thread, is);
break;
default:
break;
}
}This is pretty much the same as the code we had before, except now it's generalized for audio and video. Notice that instead of aCodecCtx, we've set up our big struct as the userdata for our audio callback. We've also saved the streams themselves as audio_st and video_st. We also have added our video queue and set it up in the same way we set up our audio queue. Most of the point is to launch the video and audio threads. These bits do it:
SDL_PauseAudio();
break;
/* ...... */
is->video_tid = SDL_CreateThread(video_thread, is);
We remember SDL_PauseAudio() from last time, and SDL_CreateThread() is used as in the exact same way as before. We'll get back to our video_thread() function.
Before that, let's go back to the second half of our decode_thread() function. It's basically just a for loop that will read in a packet and put it on the right queue:
for(;;) {
if(is->quit) {
break;
}
// seek stuff goes here
if(is->audioq.size > MAX_AUDIOQ_SIZE ||
is->videoq.size > MAX_VIDEOQ_SIZE) {
SDL_Delay();
continue;
}
if(av_read_frame(is->pFormatCtx, packet) < ) {
if(url_ferror(&pFormatCtx->pb) == ) {
SDL_Delay(); /* no error; wait for user input */
continue;
} else {
break;
}
}
// Is this a packet from the video stream?
if(packet->stream_index == is->videoStream) {
packet_queue_put(&is->videoq, packet);
} else if(packet->stream_index == is->audioStream) {
packet_queue_put(&is->audioq, packet);
} else {
av_free_packet(packet);
}
}
这里没有什么新东西,除了我们给音频和视频队列限定了一个最大值并且我们添加一个检测读错误的函数。格式上下文里面有一个叫做pb的 ByteIOContext类型结构体。这个结构体是用来保存一些低级的文件信息。函数url_ferror用来检测结构体并发现是否有些读取文件错误。
在循环以后,我们的代码是用等待其余的程序结束和提示我们已经结束的。这些代码是有益的,因为它指示出了如何驱动事件--后面我们将显示影像。
while(!is->quit) {
SDL_Delay();
}
fail:
if(){
SDL_Event event;
event.type = FF_QUIT_EVENT;
event.user.data1 = is;
SDL_PushEvent(&event);
}
return ;
我们使用SDL常量SDL_USEREVENT来从用户事件中得到值。第一个用户事件的值应当是SDL_USEREVENT,下一个是 SDL_USEREVENT+1并且依此类推。在我们的程序中FF_QUIT_EVENT被定义成SDL_USEREVENT+2。如果喜欢,我们也可以 传递用户数据,在这里我们传递的是大结构体的指针。最后我们调用SDL_PushEvent()函数。在我们的事件分支中,我们只是像以前放入 SDL_QUIT_EVENT部分一样。我们将在自己的事件队列中详细讨论,现在只是确保我们正确放入了FF_QUIT_EVENT事件,我们将在后面捕 捉到它并且设置我们的退出标志quit。
得到帧:video_thread
当我们准备好解码器后,我们开始视频线程。这个线程从视频队列中读取包,把它解码成视频帧,然后调用queue_picture函数把处理好的帧放入到图片队列中:
int video_thread(void *arg) {
VideoState *is = (VideoState *)arg;
AVPacket pkt1, *packet = &pkt1;
int len1, frameFinished;
AVFrame *pFrame;
pFrame = avcodec_alloc_frame();
for(;;) {
if(packet_queue_get(&is->videoq, packet, ) < ) {
// means we quit getting packets
break;
}
// Decode video frame
len1 = avcodec_decode_video(is->video_st->codec, pFrame, &frameFinished,
packet->data, packet->size);
// Did we get a video frame?
if(frameFinished) {
if(queue_picture(is, pFrame) < ) {
break;
}
}
av_free_packet(packet);
}
av_free(pFrame);
return ;
}
在这里的很多函数应该很熟悉吧。我们把avcodec_decode_video函数移到了这里,替换了一些参数,例如:我们把AVStream保存在我 们自己的大结构体中,所以我们可以从那里得到编解码器的信息。我们仅仅是不断的从视频队列中取包一直到有人告诉我们要停止或者出错为止。
把帧队列化
让我们看一下保存解码后的帧pFrame到图像队列中去的函数。因为我们的图像队列是SDL的覆盖的集合(基本上不用让视频显示函数再做计算了),我们需要把帧转换成相应的格式。我们保存到图像队列中的数据是我们自己做的一个结构体。
typedef struct VideoPicture {
SDL_Overlay *bmp;
int width, height;
int allocated;
} VideoPicture;
我们的大结构体有一个可以保存这些缓冲区。然而,我们需要自己来申请SDL_Overlay(注意:allocated标志会指明我们是否已经做了这个申请的动作与否)。
为了使用这个队列,我们有两个指针--写入指针和读取指针。我们也要保证一定数量的实际数据在缓冲中。要写入到队列中,我们先要等待缓冲清空以便于有位置 来保存我们的VideoPicture。然后我们检查看我们是否已经申请到了一个可以写入覆盖的索引号。如果没有,我们要申请一段空间。我们也要重新申请 缓冲如果窗口的大小已经改变。然而,为了避免被锁定,尽量避免在这里申请(我现在还不太清楚原因;我相信是为了避免在其它线程中调用SDL覆盖函数的原 因)。
int queue_picture(VideoState *is, AVFrame *pFrame) {
VideoPicture *vp;
int dst_pix_fmt;
AVPicture pict;
SDL_LockMutex(is->pictq_mutex);
while(is->pictq_size >= VIDEO_PICTURE_QUEUE_SIZE &&
!is->quit) {
SDL_CondWait(is->pictq_cond, is->pictq_mutex);
}
SDL_UnlockMutex(is->pictq_mutex);
if(is->quit)
return -;
// windex is set to 0 initially
vp = &is->pictq[is->pictq_windex];
if(!vp->bmp ||
vp->width != is->video_st->codec->width ||
vp->height != is->video_st->codec->height) {
SDL_Event event;
vp->allocated = ;
event.type = FF_ALLOC_EVENT;
event.user.data1 = is;
SDL_PushEvent(&event);
SDL_LockMutex(is->pictq_mutex);
while(!vp->allocated && !is->quit) {
SDL_CondWait(is->pictq_cond, is->pictq_mutex);
}
SDL_UnlockMutex(is->pictq_mutex);
if(is->quit) {
return -;
}
}
这里的事件机制与前面我们想要退出的时候看到的一样。我们已经定义了事件FF_ALLOC_EVENT作为SDL_USEREVENT。我们把事件发到事件队列中然后等待申请内存的函数设置好条件变量。
让我们来看一看如何来修改事件循环:
for(;;) {
SDL_WaitEvent(&event);
switch(event.type) {
case FF_ALLOC_EVENT:
alloc_picture(event.user.data1);
break;
记住event.user.data1是我们的大结构体。就这么简单。让我们看一下alloc_picture()函数:
void alloc_picture(void *userdata) {
VideoState *is = (VideoState *)userdata;
VideoPicture *vp;
vp = &is->pictq[is->pictq_windex];
if(vp->bmp) {
// we already have one make another, bigger/smaller
SDL_FreeYUVOverlay(vp->bmp);
}
// Allocate a place to put our YUV image on that screen
vp->bmp = SDL_CreateYUVOverlay(is->video_st->codec->width,
is->video_st->codec->height,
SDL_YV12_OVERLAY,
screen);
vp->width = is->video_st->codec->width;
vp->height = is->video_st->codec->height;
SDL_LockMutex(is->pictq_mutex);
vp->allocated = ;
SDL_CondSignal(is->pictq_cond);
SDL_UnlockMutex(is->pictq_mutex);
}
你可以看到我们把SDL_CreateYUVOverlay函数从主循环中移到了这里。这段代码应该完全可以自我注释。记住我们把高度和宽度保存到VideoPicture结构体中因为我们需要保存我们的视频的大小没有因为某些原因而改变。
好,我们几乎已经全部解决并且可以申请到YUV覆盖和准备好接收图像。让我们回顾一下queue_picture并看一个拷贝帧到覆盖的代码。你应该能认出其中的一部分:
int queue_picture(VideoState *is, AVFrame *pFrame) {
if(vp->bmp) {
SDL_LockYUVOverlay(vp->bmp);
dst_pix_fmt = PIX_FMT_YUV420P;
pict.data[] = vp->bmp->pixels[];
pict.data[] = vp->bmp->pixels[];
pict.data[] = vp->bmp->pixels[];
pict.linesize[] = vp->bmp->pitches[];
pict.linesize[] = vp->bmp->pitches[];
pict.linesize[] = vp->bmp->pitches[];
// Convert the image into YUV format that SDL uses
img_convert(&pict, dst_pix_fmt,
(AVPicture *)pFrame, is->video_st->codec->pix_fmt,
is->video_st->codec->width, is->video_st->codec->height);
SDL_UnlockYUVOverlay(vp->bmp);
if(++is->pictq_windex == VIDEO_PICTURE_QUEUE_SIZE) {
is->pictq_windex = ;
}
SDL_LockMutex(is->pictq_mutex);
is->pictq_size++;
SDL_UnlockMutex(is->pictq_mutex);
}
return ;
}
这部分代码和前面用到的一样,主要是简单的用我们的帧来填充YUV覆盖。最后一点只是简单的给队列加1。这个队列在写的时候会一直写入到满为止,在读的时 候会一直读空为止。因此所有的都依赖于is->pictq_size值,这要求我们必需要锁定它。这里我们做的是增加写指针(在必要的时候采用轮转 的方式),然后锁定队列并且增加尺寸。现在我们的读者函数将会知道队列中有了更多的信息,当队列满的时候,我们的写入函数也会知道。
FFPLAY的原理(五)的更多相关文章
- How Javascript works (Javascript工作原理) (五) 深入理解 WebSockets 和带有 SSE 机制的HTTP/2 以及正确的使用姿势
个人总结: 1.长连接机制——分清Websocket,http2,SSE: 1)HTTP/2 引进了 Server Push 技术用来让服务器主动向客户端缓存发送数据.然而,它并不允许直接向客户端程序 ...
- FFPLAY的原理
概要 电影文件有很多基本的组成部分.首先,文件本身被称为容器Container,容器的类型决定了信息被存放在文件中的位置.AVI和Quicktime就是容器的例子.接着,你有一组流,例如,你经常有的是 ...
- FFPLAY的原理(六)
显示视频 这就是我们的视频线程.现在我们看过了几乎所有的线程除了一个--记得我们调用schedule_refresh()函数吗?让我们看一下实际中是如何做的: static void schedule ...
- FFPLAY的原理(三)
播放声音 现在我们要来播放声音.SDL也为我们准备了输出声音的方法.函数SDL_OpenAudio()本身就是用来打开声音设备的.它使用一个叫做SDL_AudioSpec结构体作为参数,这个结构体中包 ...
- FFPLAY的原理(七)
同步音频 现在我们已经有了一个比较像样的播放器.所以让我们看一下还有哪些零碎的东西没处理.上次,我们掩饰了一点同步问题,也就是同步音频到视频而不是其它的同 步方式.我们将采用和视频一样的方式:做一个内 ...
- FFPLAY的原理(一)
概要 电影文件有很多基本的组成部分.首先,文件本身被称为容器Container,容器的类型决定了信息被存放在文件中的位置.AVI和Quicktime就 是容器的例子.接着,你有一组流,例如,你经常有的 ...
- redis实现cache系统原理(五)
1. 介绍 cache就是人们所说的缓存.我们这里所说的cache是web上的.对用户来说,衡量一个网站是否具有良好的体验,其中一个标准就是响应速度的快慢.可能网站刚上线,功能还较少,数据库的记录也不 ...
- FFPLAY的原理(四)
意外情况 你们将会注意到我们有一个全局变量quit,我们用它来保证还没有设置程序退出的信号(SDL会自动处理TERM类似的信号).否则,这个线程将不停地运 行直到我们使用kill -9来结束程序.FF ...
- FFPLAY的原理(二)
关于包Packets的注释 从技术上讲一个包可以包含部分或者其它的数据,但是ffmpeg的解释器保证了我们得到的包Packets包含的要么是完整的要么是多种完整的帧. 现在我们需要做的是让SaveFr ...
随机推荐
- Xcode中Groups和Folder的区别
以前一直没有在意这个问题,今天"中奖"了 ;( 在Xcode文件夹视图中会发现2种颜色的文件夹图标:黄色和蓝色. 黄色表示组(groups),蓝色表示文件夹(folder). 组只 ...
- 常用js总结
通过radio来显示/隐藏一个div <div id="ArbAcei" > <br/> 有无不良反应停药 <input type="rad ...
- Intent的Data和Type和Flag属性-amdroid学习之旅(五十一)
Data属性介绍 使用Data和Action属性启动系统Activity 代码示例 public class MainActivity extends Activity{ @Override prot ...
- Uva - 1598 - Exchange
本来想用优先队列做,可是不知道怎么处理之间的关系,最后还是用了map方法AC了,不过速度上有些慢,提交的时候跑了1.557秒.估计这道题时间都稍微长些,题目的时间限制也是4.5秒,不像一般题目的3秒限 ...
- Mybatis插件原理分析(一)
我们首先介绍一下Mybatis插件相关的几个类,并对源码进行了简单的分析. Mybatis插件相关的接口或类有:Intercept.InterceptChain.Plugin和Invocation,这 ...
- 04-GIT TortoiseGit冲突和补丁演示 案例演示
TortoiseGit安装下载 http://download.tortoisegit.org/tgit/1.8.12.0/ 或https://code.google.com/p/tortoisegi ...
- cocos2d-x初探
今天把cocos2d-x下载下来装了准备试试. 就不用windows本了,主要想做iOS游戏,所以这里用mac. 先在http://cocos2d-x.org/download下载右边的cocos,然 ...
- C++ Primer 有感(异常处理)(三)
先看下面的代码: [cpp] view plaincopy int main() { int *i=new int(10); /* 这中间的代码出现异常 */ delete i; return 0; ...
- shell脚本格式的几点注意:格式严格,空格不能随便出现(一写就记不住)
shell脚本中,不能随意添加空格,否则出错: 1,=等号两边必须无空格.否则出错.如i =$1和i= $1都是错的.但是在()内部不限制如for ((i= 1;i < 3;i= i+1))是正 ...
- 《java入门第一季》之面向对象(抽象类其实不抽象)
上一篇(http://blog.csdn.net/qq_32059827/article/details/51334198)对抽象类做了一些描述,这一篇根据两个案例加深一下印象.会觉得抽象类其实不抽象 ...