SignalR+AForge实现视频会话[WPF]

原文:SignalR+AForge实现视频会话[WPF]

版权声明：本文为博主原创文章，未经博主允许不得转载。 https://blog.csdn.net/lordwish/article/details/51785023

AForge是基于.NET的强大视频分析库，而SignalR是微软推出的实时通信技术，两者结合起来实现简单的视频会话。预期的效果是实现：在线终端刷新、会话请求、会话接受、会话拒绝、会话繁忙、会话结束。本示例采用USB摄像头。

---

1服务端及辅助类

1.1创建服务端

[ 使用WPF创建SignalR服务端]

1.2在线终端刷新

[ WPF+SignalR实现用户列表实时刷新]

1.3通信状态辅助类

    public enum MessageState
    {
        VideoApply,//视频会话请求
        VideoOpen,//视频会话开始
        VideoRefuse,//视频会话拒绝
        VideoBusy,//视频会话繁忙
        VideoOver,//视频会话结束
        Null
    }

1.4服务端方法

在MyHub类中添加方法MessageConnect，其中参数messageid是由会话发起端生成的唯一Guid,用来标识此次会话，当会话接受端拒绝、繁忙或会话过程中任一方终止会话时，会话结束，移除messageid。因为会话状态消息在两个终端之间来回传输的，所以从消息的角度讲，终端发送消息时就是sender，接收消息时就是receiver了。

public Task MessageConnect(ClientModel sender,ClientModel receiver,string messageid,MessageState status)
        {
            return Clients.Client(receiver.ConnectionId).messageconnect(receiver, sender, messageid, status);
        }

2客户端方法

使用NuGet添加AForge程序包，只安装AForge.Video和AForge.Controls就够了，图方便全装也可以。

2.1页面布局

摄像头图像显示使用AForge.Controls中的控件VideoSourcePlayer，所以要引用AForge.Controls命名空间。由于视频实际上是以图像帧的形式传输的，格式为bitmap，所以对方图像显示在PictureBox中，对System.Windows.Form的引用也是不能少的。

    xmlns:aforge="clr-namespace:AForge.Controls;assembly=AForge.Controls"
    xmlns:form="clr-namespace:System.Windows.Forms;assembly=System.Windows.Forms"

        <WrapPanel>
            <Grid Margin="10,10,5,10" VerticalAlignment="Center" HorizontalAlignment="Center">
                <Grid.RowDefinitions>
                    <RowDefinition></RowDefinition>
                    <RowDefinition Height="50"></RowDefinition>
                </Grid.RowDefinitions>
                <WindowsFormsHost  Width="320" Height="240" VerticalAlignment="Center" HorizontalAlignment="Center">
                    <form:PictureBox x:Name="imgCapture" Width="320" Height="240"></form:PictureBox>
                </WindowsFormsHost>
                <StackPanel Grid.Row="1" Orientation="Horizontal" VerticalAlignment="Center" HorizontalAlignment="Center">
                    <TextBlock Name="txtName"></TextBlock>
                </StackPanel>
            </Grid>
            <Grid Margin="5,10,10,10" VerticalAlignment="Center" HorizontalAlignment="Center">
                <Grid.RowDefinitions>
                    <RowDefinition></RowDefinition>
                    <RowDefinition Height="50"></RowDefinition>
                </Grid.RowDefinitions>
                <WindowsFormsHost Width="320" Height="240" VerticalAlignment="Center" HorizontalAlignment="Center">
                    <aforge:VideoSourcePlayer x:Name="sourcePlayer" Width="320" Height="240"></aforge:VideoSourcePlayer>
                </WindowsFormsHost>
                <StackPanel Grid.Row="1" VerticalAlignment="Center" Orientation="Horizontal" HorizontalAlignment="Center">
                    <Button Name="btnCapture" Width="90" Height="30" Click="btnCapture_Click">捕获</Button>
                    <Button Name="btnClose" Width="90" Height="30" Margin="10,0,0,0" Click="btnClose_Click">关闭</Button>
                </StackPanel>
            </Grid>
        </WrapPanel>

2.2发送会话请求

选择在线终端列表上的任一用户，双击发起会话请求。

        private void dgList_MouseDoubleClick(object sender, MouseButtonEventArgs e)
        {
            if (dgList.SelectedItem == null)
            {
                return;
            }
            client = (ClientModel)dgList.SelectedItem;
            var result = MessageBox.Show("确定要进行视频会话？", "会话提醒", MessageBoxButton.OKCancel, MessageBoxImage.Question, MessageBoxResult.OK);
            if (result == MessageBoxResult.OK && client != null)
            {
                string messageid = Guid.NewGuid().ToString();
                MyHub.Invoke("MessageConnect", MyClient, client, messageid, MessageState.VideoApply);
            }
        }

2.3会话状态监听

接收到会话请求时，首先会判断本地是不是已经有正在进行的视频会话了，如果有就返回繁忙标识，没有就打开摄像头。这里对会话关闭状态是为了在一方终止会话关闭摄像头时，另一方也能收到关闭通知从而关闭摄像头。

        private void ConnectListener()
        {
            try
            {
                MyHub.On<ClientModel, ClientModel, string, MessageState>("messageconnect", (sender, receiver, id, state) =>
                {
                    switch (state)
                    {
                        case MessageState.VideoApply:
                            {
                                if (messageid == "")
                                {
                                    var result = MessageBox.Show("是否接收" + sender.ClientName + "的会话请求？", "会话提醒", MessageBoxButton.YesNo, MessageBoxImage.Question, MessageBoxResult.Yes);
                                    if (result == MessageBoxResult.Yes)
                                    {
                                        this.Dispatcher.Invoke(delegate
                                        {
                                            client = sender;
                                            messageid = id;
                                            txtName.Text = sender.ClientName;
                                            OpenVideo();//打开摄像头
                                        });
                                    }
                                    else
                                    {
                                        MyHub.Invoke("MessageConnect", receiver, sender, id, MessageState.VideoRefuse);
                                    }
                                }
                                else
                                {
                                    MyHub.Invoke("MessageConnect", receiver, sender, id, MessageState.VideoBusy);
                                }
                            }
                            break;
                        case MessageState.VideoOpen:
                            {
                                this.Dispatcher.Invoke(delegate
                                {
                                    client = sender;
                                    messageid = id;
                                    txtName.Text = sender.ClientName;
                                    OpenVideo();//打开摄像头
                                });
                            }
                            break;
                        case MessageState.VideoRefuse:
                            {
                                MessageBox.Show(sender.ClientName + "拒绝了您的视频会话请求！");
                            }
                            break;
                        case MessageState.VideoBusy:
                            {
                                MessageBox.Show(sender.ClientName + "正忙，请稍后发送请求！");
                            }
                            break;
                        case MessageState.VideoOver:
                            {
                                this.Dispatcher.Invoke(delegate
                                {
                                    CloseVideo();//关闭摄像头
                                });
                            }
                            break;
                    }
                });
            }
            catch (Exception)
            {
                throw;
            }
        }

2.4打开摄像头

在打开摄像头后，为控件绑定NewFrame事件开始发送图像帧，同时开启线程videoThread接收图像帧，并显示在PictureBox中。发送和接收其实都是用了最基本的Socket。

        private Socket videoSocket;
        private Thread videoThread;
        private object bitmapTemp = new object();
        private IPEndPoint receiver = null;
        private byte[] buffTemp = new byte[2 * 1024 * 1024];
        private void OpenVideo()
        {
            try
            {
                receiver = new IPEndPoint(IPAddress.Parse(client.ClientIP), VideoPort);
                videoSocket = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
                uint IOC_IN = 0x80000000;
                uint IOC_VENDOR = 0x18000000;
                uint SIO_UDP_CONNRESET = IOC_IN | IOC_VENDOR | 12;
                videoSocket.IOControl((int)SIO_UDP_CONNRESET, new byte[] { Convert.ToByte(false) }, null);
                videoSocket.Bind(new IPEndPoint(IPAddress.Parse(ClientIp), VideoPort));

                FilterInfoCollection videoDevices = new FilterInfoCollection(FilterCategory.VideoInputDevice);
                if (videoDevices.Count > 0)
                {
                    sourcePlayer.VideoSource = new VideoCaptureDevice(videoDevices[0].MonikerString);
                    sourcePlayer.NewFrame += SourcePlayer_NewFrame;
                    sourcePlayer.Start();
                    videoThread = new Thread(ReceiveVideo);
                    videoThread.IsBackground = true;
                    videoThread.Start();
                }
                else
                {
                    MessageBox.Show("未找到视频设备！");
                }

            }
            catch (Exception ex)
            {
                MessageBox.Show(ex.Message);
                //throw;
            }
        }

2.5发送图像帧

        private void SourcePlayer_NewFrame(object sender, ref Bitmap image)
        {
            lock (bitmapTemp)
            {
                System.Drawing.Image imageTemp = image.Clone(new System.Drawing.Rectangle(0, 0, image.Width, image.Height), image.PixelFormat);
                MemoryStream stream = new MemoryStream();
                imageTemp.Save(stream, System.Drawing.Imaging.ImageFormat.Jpeg);
                stream.Position = 0;
                byte[] buffImage = new byte[stream.Length];
                stream.Read(buffImage, 0, buffImage.Length);
                videoSocket.BeginSendTo(buffImage, 0, buffImage.Length, SocketFlags.None, receiver, SendData, videoSocket);
                stream.Dispose();
                stream = null;
            }
        }
        private void SendData(IAsyncResult ar)
        {
            Socket socket = (Socket)ar.AsyncState;
            socket.EndSendTo(ar);
        }

2.6接收图像帧

        private void ReceiveVideo()
        {
            while (true)
            {
                bool result = videoSocket.Poll(5000, SelectMode.SelectRead);
                if (result)
                {
                    videoSocket.BeginReceive(buffTemp, 0, buffTemp.Length, SocketFlags.None, ReceiveData, videoSocket);
                }
            }
        }
        private void ReceiveData(IAsyncResult ar)
        {
            Socket socket = (Socket)ar.AsyncState;
            int count = socket.EndReceive(ar);
            if (count > 0)
            {
                byte[] buff = new byte[count];
                Buffer.BlockCopy(buffTemp, 0, buff, 0, count);
                MemoryStream ms = new MemoryStream(buff);
                Bitmap bitmap = (Bitmap)System.Drawing.Image.FromStream(ms);
                imgCapture.SizeMode = System.Windows.Forms.PictureBoxSizeMode.StretchImage;
                imgCapture.Image = bitmap;
            }
        }

2.7关闭摄像头

        private void btnClose_Click(object sender, RoutedEventArgs e)
        {
            try
            {
                MyHub.Invoke("MessageConnect", MyClient, client, messageid, MessageState.VideoOver);
                CloseVideo();
                client = null;
                messageid = "";

            }
            catch (Exception)
            {

                throw;
            }
        }
        private void CloseVideo()
        {
            videoThread.Abort();
            if (sourcePlayer.IsRunning)
            {
                sourcePlayer.SignalToStop();
                sourcePlayer.WaitForStop();
            }
            imgCapture.Image = null;
            sourcePlayer.NewFrame -= SourcePlayer_NewFrame;
        }

---

后记

总体来说运行效果还是不错的。最开始想用SignalR做图像帧的发送和接收，发现没经过压缩的图像数据量太大，SignalR根本反应不过来，还是用底层的Socket快。而SignalR中的很多特性都是从底层封装好的，在实时通信上比Socket要方便很多。此外视频控件并不局限于AForge，类似WPFMediaTookit之类的也可以做，原理都是一样的。