利用html5、websocket和opencv实现人脸检测

最近学习人脸识别相关的东西，在MFC下使用OpenCV做了一个简单的应用。训练需要较多的数据，windows应用程序终究还是不方便，于是想着做成CS模式：检测识别都放在服务器端，视频获取和显示都放在网页端。

在网上找了一些资料，实现了简单的人脸检测。人脸识别只要在这个框架上加点代码就行。主要参考了下面这篇文章：

http://www.open-open.com/home/space-361-do-blog-id-8960.html

jetty版本：jetty-9.2.17.v20160517

javacv版本：1.2

首先是html代码，主要实现：

获取视频并显示（html5, webrtc, javascript）；
通过websocket传输视频帧；
接收并显示服务器端返回的图像数据（包含人脸检测结果）

 <!doctype html>

 <html lang="zh-CN">

 <head>

     <meta charset="UTF-8">

     <title>FaceDetect</title>

 </head>

 <body>

 <div style="visibility:hidden; width:0; height:0;">

     <canvas id="canvas" width="320"  height="240"></canvas>

 </div>

 <div>

     <video id="video" autoplay style="display: inline;"></video>

     <img id="target" style="display:inline;"/>

 </div>

 <script type="text/javascript">

     var ws = new WebSocket("ws://127.0.0.1:2014/");

     ws.binaryType = "arraybuffer";

     ws.onopen = function() {

         ws.send("I'm client");

     };

     ws.onmessage = function (evt) {

         var bytes = new Uint8Array(evt.data);

         var data = "";

         var len = bytes.byteLength;

         for (var i = 0; i < len; ++i) {

             data += String.fromCharCode(bytes[i]);

         }

         var img = document.getElementById("target");

         img.src = "data:image/png;base64,"+window.btoa(data);

     };

     ws.onclose = function() {

         alert("Closed");

     };

     ws.onerror = function(err) {

         alert("Error: " + err);

     };

     var getUserMedia = (navigator.getUserMedia || navigator.webkitGetUserMedia || navigator.mozGetUserMedia || navigator.msGetUserMedia);

     var video = document.getElementById('video');

     var canvas = document.getElementById('canvas');

     var ctx = canvas.getContext('2d');

     getUserMedia.call(navigator, {

         video: true,

         audio: true

     }, function(localMediaStream) {

         video.src = window.URL.createObjectURL(localMediaStream);

         video.onloadedmetadata = function(e) {

             console.log("Label: " + localMediaStream.label);

             console.log("AudioTracks" , localMediaStream.getAudioTracks());

             console.log("VideoTracks" , localMediaStream.getVideoTracks());

         };

     }, function(e) {

         console.log('Reeeejected!', e);

     });

     function dataURItoBlob(dataURI) {

         // convert base64/URLEncoded data component to raw binary data held in a string

         var byteString;

         if (dataURI.split(',')[0].indexOf('base64') >= 0)

             byteString = atob(dataURI.split(',')[1]);

         else

             byteString = unescape(dataURI.split(',')[1]);

         // separate out the mime component

         var mimeString = dataURI.split(',')[0].split(':')[1].split(';')[0];

         // write the bytes of the string to a typed array

         var ia = new Uint8Array(byteString.length);

         for (var i = 0; i < byteString.length; i++) {

             ia[i] = byteString.charCodeAt(i);

         }

         return new Blob([ia], {type:mimeString});

     }

     timer = setInterval(

             function () {

                 ctx.drawImage(video, 0, 0, 320, 240);

                 var data = canvas.toDataURL('image/jpeg', 1.0);

                 newblob = dataURItoBlob(data);

                 ws.send(newblob);

             }, 250);

 </script>

 </body>

 </html>

facedetect.html

然后是服务器端代码（jetty, websocket, javacv），主要实现：

接收客户传送的视频帧数据；
使用JavaCV实现人脸检测；
在原始图像上绘制检测结果，将新图像返回给客户

 package com.husthzy.face;

 import org.eclipse.jetty.server.Server;

 public class WebsocketServer extends Thread {

     @Override

     public void run() {

         super.run();

         try {

             Server server = new Server(2014);

             server.setHandler(new FaceDetectionHandler());

             server.setStopTimeout(0);

             server.start();

             server.join();

         } catch (Exception e) {

             e.printStackTrace();

         }

     }

     public static void main(String[] args) {

         WebsocketServer mWebSocketServer = new WebsocketServer();

         mWebSocketServer.start();

     }

 }

WebsocketServer.java

 package com.husthzy.face;

 import static org.bytedeco.javacpp.opencv_core.CV_8UC1;

 import static org.bytedeco.javacpp.opencv_imgcodecs.cvDecodeImage;

 import static org.bytedeco.javacpp.opencv_imgproc.COLOR_BGRA2GRAY;

 import static org.bytedeco.javacpp.opencv_imgproc.cvtColor;

 import static org.bytedeco.javacpp.opencv_imgproc.equalizeHist;

 import static org.bytedeco.javacpp.opencv_imgproc.rectangle;

 import java.awt.image.BufferedImage;

 import java.io.ByteArrayOutputStream;

 import java.io.IOException;

 import java.nio.ByteBuffer;

 import java.util.ArrayList;

 import javax.imageio.ImageIO;

 import org.bytedeco.javacpp.BytePointer;

 import org.bytedeco.javacpp.opencv_core;

 import org.bytedeco.javacpp.opencv_core.IplImage;

 import org.bytedeco.javacpp.opencv_core.Mat;

 import org.bytedeco.javacpp.opencv_core.Rect;

 import org.bytedeco.javacpp.opencv_core.RectVector;

 import org.bytedeco.javacpp.opencv_core.Scalar;

 import org.bytedeco.javacpp.opencv_objdetect.CascadeClassifier;

 import org.bytedeco.javacv.Frame;

 import org.bytedeco.javacv.Java2DFrameConverter;

 import org.bytedeco.javacv.OpenCVFrameConverter.ToMat;

 import org.eclipse.jetty.websocket.api.Session;

 import org.eclipse.jetty.websocket.api.annotations.OnWebSocketClose;

 import org.eclipse.jetty.websocket.api.annotations.OnWebSocketConnect;

 import org.eclipse.jetty.websocket.api.annotations.OnWebSocketError;

 import org.eclipse.jetty.websocket.api.annotations.OnWebSocketMessage;

 import org.eclipse.jetty.websocket.api.annotations.WebSocket;

 import org.eclipse.jetty.websocket.server.WebSocketHandler;

 import org.eclipse.jetty.websocket.servlet.WebSocketServletFactory;

 @WebSocket

 public class FaceDetectionHandler extends WebSocketHandler {

     private static final String CASCADE_FILE = "haarcascade_frontalface_alt.xml";

     private Session mSession;

     private static ArrayList<FaceDetectionHandler> sessions = new ArrayList<FaceDetectionHandler>();

     private CascadeClassifier face_cascade = new CascadeClassifier(CASCADE_FILE);

     public static ArrayList<FaceDetectionHandler> getAllSessions() {

         return sessions;

     }

     @Override

     public void configure(WebSocketServletFactory factory) {

         factory.register(FaceDetectionHandler.class);

         factory.getPolicy().setMaxBinaryMessageSize(1024 * 512);

     }

     @OnWebSocketClose

     public void onClose(int statusCode, String reason) {

         sessions.remove(this);

         System.out.println(

                 "Close: statusCode = " + statusCode + ", reason = " + reason + ", sessions = " + sessions.size());

     }

     @OnWebSocketError

     public void onError(Throwable t) {

         System.out.println("Error: " + t.getMessage());

     }

     @OnWebSocketConnect

     public void onConnect(Session session) {

         mSession = session;

         sessions.add(this);

         System.out.println("Connect: " + session.getRemoteAddress().getAddress());

     }

     @OnWebSocketMessage

     public void onMessage(String message) {

         System.out.println("Message: " + message);

     }

     @OnWebSocketMessage

     public void onBinaryMessage(byte data[], int offset, int length) {

         System.out.println("Binary Message len:" + length);

         if (length > 10000) {

             try {

                 byte[] sdata = process(data);

                 ByteBuffer byteBuffer = ByteBuffer.wrap(sdata);

                 mSession.getRemote().sendBytes(byteBuffer);

                 byteBuffer.clear();

             } catch (IOException e) {

                 e.printStackTrace();

             }

         }

     }

     public byte[] process(byte data[]) {

         IplImage originalImage = cvDecodeImage(opencv_core.cvMat(1, data.length, CV_8UC1, new BytePointer(data)));

         Mat videoMat = new Mat(originalImage);

         Mat videoMatGray = new Mat();

         // Convert the current frame to grayscale:

         cvtColor(videoMat, videoMatGray, COLOR_BGRA2GRAY);

         equalizeHist(videoMatGray, videoMatGray);

         // Point p = new Point();

         RectVector faces = new RectVector();

         face_cascade.detectMultiScale(videoMatGray, faces);

         for (int i = 0; i < faces.size(); i++) {

             Rect face_i = faces.get(i);

             //Mat face = new Mat(videoMatGray, face_i);

             // If fisher face recognizer is used, the face need to be

             // resized.

             // resize(face, face_resized, new Size(im_width, im_height),

             // 1.0, 1.0, INTER_CUBIC);

             // Now perform the prediction, see how easy that is:

             // int prediction = lbphFaceRecognizer.predict(face);

             // And finally write all we've found out to the original image!

             // First of all draw a green rectangle around the detected face:

             rectangle(videoMat, face_i, new Scalar(0, 255, 0, 1));

             System.out.println("face pos: x:" + face_i.x() + " y:" + face_i.y());

             // Create the text we will annotate the box with:

             //String box_text = "Prediction = " + prediction;

             // Calculate the position for annotated text (make sure we don't

             // put illegal values in there):

             //int pos_x = Math.max(face_i.tl().x() - 10, 0);

             //int pos_y = Math.max(face_i.tl().y() - 10, 0);

             // And now put it into the image:

             //putText(videoMat, box_text, new Point(pos_x, pos_y), FONT_HERSHEY_PLAIN, 1.0, new Scalar(0, 255, 0, 2.0));

         }

         // JavaCVUtil.imShow(videoMat, "test");

         return getMatByteBuffer(videoMat);

     }

     private byte[] getMatByteBuffer(Mat m) {

         byte[] result = null;

         try {

             ToMat convert = new ToMat();

             Frame frame = convert.convert(m);

             Java2DFrameConverter java2dFrameConverter = new Java2DFrameConverter();

             BufferedImage bufferedImage = java2dFrameConverter.convert(frame);

             ByteArrayOutputStream out = new ByteArrayOutputStream();

             ImageIO.write(bufferedImage, "png", out);

             result = out.toByteArray();

             out.close();

         } catch (IOException exception) {

             exception.printStackTrace();

         }

         return result;

     }

 }

FaceDetectionhandler.java

利用html5、websocket和opencv实现人脸检测的更多相关文章

利用html5、websocket和opencv实现人脸检测（二）
前一篇的代码在执行时,java.exe占用内存会快速上涨: 在4G内存电脑上,单个连接,会持续上涨到2G多,然后减到1G多,如此循环. 经过一些删减定位,可以确定问题由public byte[] pr ...
OpenCV实现人脸检测
OpenCV实现人脸检测(转载) 原文链接:https://www.cnblogs.com/mengdd/archive/2012/08/01/2619043.html 本文介绍最基本的用OpenC ...
【转载】opencv实现人脸检测
全文转载自CSDN的博客(不知道怎么将CSDN的博客转到博客园,应该没这功能吧,所以直接复制全文了),转载地址如下 http://blog.csdn.net/lsq2902101015/article ...
OpenCV学习系列（一） Mac下OpenCV + xcode人脸检测实现
# OpenCV学习系列(一) Mac下OpenCV + xcode人脸检测实现 [-= 博客目录 =-] 1-学习目标 1.1-本章介绍 1.2-实践内容 1.3-相关说明 2-学习过程 2.1-环 ...
利用OpenCV的人脸检测给头像带上圣诞帽
我们来看下效果原图: 效果: 原理其实很简单: 采用一张圣诞帽的png图像作为素材, 利用png图像背景是透明的,贴在背景图片上就是戴帽子的效果了. 人脸检测的目的主要是为了确定贴帽子的位置,类似p ...
基于OpenCv的人脸检测、识别系统学习制作笔记之三
1.在windows下编写人脸检测.识别系统.目前已完成:可利用摄像头提取图像,并将人脸检测出来,未进行识别. 2.在linux下进行编译在windows环境下已经能运行的代码. 为此进行了linux ...
OpenCV&Qt学习之四——OpenCV 实现人脸检测与相关知识整理
开发配置 OpenCV的例程中已经带有了人脸检测的例程,位置在:OpenCV\samples\facedetect.cpp文件,OpenCV的安装与这个例子的测试可以参考我之前的博文Linux 下编译 ...
基于OpenCv的人脸检测、识别系统学习制作笔记之一
基于OpenCv从视频文件到摄像头的人脸检测在OpenCv中读取视频文件和读取摄像头的的视频流然后在放在一个窗口中显示结果其实是类似的一个实现过程. 先创建一个指向CvCapture结构的指针 Cv ...
Python学习--使用dlib、opencv进行人脸检测标注
参考自https://www.pyimagesearch.com/2017/04/03/facial-landmarks-dlib-opencv-python/ 在原有基础上有一部分的修改(image ...

随机推荐

JS 中如何将<br/> 替换成 /n
JS 中如何将<br/> 替换成 /n function a() { var data = "aaaa<br/>bbbb<br/>cccc"; ...
BZOJ2588: Spoj 10628. Count on a tree
传送门刚开始看错题以为是dfs序瞎搞.. 后来看清题了开始想用树剖瞎搞... 感觉要滚粗啊.. 对于每个点到根的路径建立线段树,暴力建MLE没跑,上主席树,然后$(x,y)$的路径就可以先求出来$L ...
form表单练习
注册页面的设计 <body leftmargin="400px" topmargin="200px"> <form method=" ...
php登录主机机制
注册机制 //用户名需过滤的字符的正则 $stripChar = '?<*.>\'"'; ){ $this->error('用户名中包含'.$stripChar.'等非法字 ...
todo
ID生成器要做成兼容分布式, 数据库ef控制要改成手动升级, 异常日志模块,操作日志某快,,, 日志服务器,, 图片服务器,,,动静分离, 前后台分离,, 可扩展性,无状态化集群弹性部署, 数据库主 ...
启动Tomcat内存溢出解决：java.lang.OutOfMemoryError: PermGen space
Eclispe 设置Tomcat的时候,双击server的配置,配置如下:
关于点击Invalidate Caches/Restart禁止插件后，重新加载--Android Studio
1:47:27 Plugin Error Problems found loading plugins: Plugin "Google Analytics Uploader" wa ...
java初始化
一.成员初始化 1.成员变量没有赋值,则被初始化成默认值. 2.局部变量没有赋值,编译时报错. 二.构造器初始化 1.成员变量在构造器初始化之前,已经被初始化. 2.变量定义的顺序决定了初始化的顺序. ...
Git 账户认证的一些问题
Mac被水浇了之后,只好用Window 开发了.但是在往Github上提交代码出现了些问题. 1. 提交时总是出现弹出框提示账号认证之前Mac没有出现过这问题,所以有些怀疑Windows开发环境不友 ...
取代 Windows Search
windows自带的搜索工具太难用了,总是在你急需的时候提示还没有建立索引,眼皮底下的文件都找不到. 1. everything 适合快速搜索文件名优点是速度快,非常快,几乎是瞬间就建立好了索引. ...

利用html5、websocket和opencv实现人脸检测

利用html5、websocket和opencv实现人脸检测的更多相关文章

随机推荐

热门专题