MQTT研究之EMQ：【SSL证书链验证】

1. 创建证书链（shell脚本）

客户端证书链关系：

rootCA-->chainca1-->chainca2-->chainca3
ca       caCert1     caCert2     caCert 

#!/bin/bash

set -e
dir=`pwd`
root_key=$dir/rootCA.key
root_crt=$dir/rootCA.crt
echo "rootKey: $root_key, rootCrt: $root_crt"

key_bits=
expire_days=
subj1=/C="CN"/ST="Hubei"/L="Wuhan"/O="Taikang"/OU="TKCloud"/CN="CAC1"
subj2=/C="CN"/ST="Hubei"/L="Wuhan"/O="Taikang"/OU="TKCloud"/CN="CAC2"
subj3=/C="CN"/ST="Hubei"/L="Wuhan"/O="Taikang"/OU="TKCloud"/CN="CAC3"
server="chainca"
param=$server
if [ -d $param ]; then
    rm -r $param
fi
mkdir -p $param
cd $param

ca1key_name=$param1.key
ca1csr_name=$param1.csr
ca1crt_name=$param1.crt
ca2key_name=$param2.key
ca2csr_name=$param2.csr
ca2crt_name=$param2.crt
ca3key_name=$param3.key
ca3csr_name=$param3.csr
ca3crt_name=$param3.crt

#cd -
#SUB1 CA
openssl genrsa -out $ca1key_name $key_bits
openssl req -new -key $ca1key_name -sha256 -out $ca1csr_name -subj $subj1 -days $expire_days
openssl ca -batch -in $ca1csr_name -out $ca1crt_name -cert $root_crt -keyfile $root_key
echo "===================Gen SUB1 CA OK===================="

#SUB2 CA
openssl genrsa -out $ca2key_name $key_bits
openssl req -new -key $ca2key_name -sha256 -out $ca2csr_name -subj $subj2 -days $expire_days
openssl ca -batch -in $ca2csr_name -out $ca2crt_name -cert $ca1crt_name -keyfile $ca1key_name
echo "===================Gen SUB2 CA OK===================="

#SUB3 CA
openssl genrsa -out $ca3key_name $key_bits
openssl req -new -key $ca3key_name -sha256 -out $ca3csr_name -subj $subj3 -days $expire_days
openssl ca -batch -in $ca3csr_name -out $ca3crt_name -cert $ca2crt_name -keyfile $ca2key_name
echo "===================Gen SUB3 CA OK===================="
rm -f *.csr

cat $root_crt $ca1crt_name $ca2crt_name |tee $param.pem
echo "===================Gen All OK===================="

2. emqttd配置/etc/emqttd/emq.conf

EMQ服务端的配置，SSL模式，参考器的前一篇博客 MQTT研究之EMQ：【SSL双向验证】

CA和客户端CA一样，rootCA，证书server.crt, server.key

## Path to the file containing the user's private PEM-encoded key.
##
## See: http://erlang.org/doc/man/ssl.html
##
## Value: File
#listener.ssl.external.keyfile = /etc/emqttd/certs/key.pem
listener.ssl.external.keyfile = /opt/certs/server.key

## Path to a file containing the user certificate.
##
## See: http://erlang.org/doc/man/ssl.html
##
## Value: File
#listener.ssl.external.certfile = /etc/emqttd/certs/cert.pem
listener.ssl.external.certfile = /opt/certs/server.crt

## Path to the file containing PEM-encoded CA certificates. The CA certificates
## are used during server authentication and when building the client certificate chain.
##
## Value: File
## listener.ssl.external.cacertfile = /etc/emqttd/certs/cacert.pem
listener.ssl.external.cacertfile = /opt/certs/rootCA.crt

## The Ephemeral Diffie-Helman key exchange is a very effective way of
## ensuring Forward Secrecy by exchanging a set of keys that never hit
## the wire. Since the DH key is effectively signed by the private key,
## it needs to be at least as strong as the private key. In addition,
## the default DH groups that most of the OpenSSL installations have
## are only a handful (since they are distributed with the OpenSSL
## package that has been built for the operating system it’s running on)
## and hence predictable (not to mention,  bits only).
## In order to escape this situation, first we need to generate a fresh,
## strong DH group, store it in a file and then use the option above,
## to force our SSL application to use the new DH group. Fortunately,
## OpenSSL provides us with a tool to do that. Simply run:
## openssl dhparam -out dh-params.pem
##
## Value: File
## listener.ssl.external.dhfile = /etc/emqttd/certs/dh-params.pem

## A server only does x509-path validation in mode verify_peer,
## as it then sends a certificate request to the client (this
## message is not sent if the verify option is verify_none).
## You can then also want to specify option fail_if_no_peer_cert.
## More information at: http://erlang.org/doc/man/ssl.html
##
## Value: verify_peer | verify_none
listener.ssl.external.verify = verify_peer

## Used together with {verify, verify_peer} by an SSL server. If set to true,
## the server fails if the client does not have a certificate to send, that is,
## sends an empty certificate.
##
## Value: true | false
listener.ssl.external.fail_if_no_peer_cert = true

3. 基于paho的java客户端（demo代码）

import com.taikang.iot.re.demo.PushCallback;
import org.eclipse.paho.client.mqttv3.MqttClient;
import org.eclipse.paho.client.mqttv3.MqttConnectOptions;
import org.eclipse.paho.client.mqttv3.MqttException;
import org.eclipse.paho.client.mqttv3.MqttTopic;
import org.eclipse.paho.client.mqttv3.persist.MemoryPersistence;

import javax.net.ssl.SSLSocketFactory;
import java.util.concurrent.ScheduledExecutorService;

public class SSLChainConsumer {
    public static final String HOST = "ssl://10.95.197.3:8883";
    public static final String TOPIC1 = "taikang/rulee";
    private static final String clientid = "client11";
    private MqttClient client;
    private MqttConnectOptions options;
    private String userName = "water";    //非必须
    private String passWord = "water";  //非必须
    @SuppressWarnings("unused")
    private ScheduledExecutorService scheduler;
    private String sslPemPath = "E:\\HOWTO\\emqtt-ssl\\self1\\";

    private void start() {
        try {
            // host为主机名，clientid即连接MQTT的客户端ID，一般以唯一标识符表示，MemoryPersistence设置clientid的保存形式，默认为以内存保存
            client = new MqttClient(HOST, clientid, new MemoryPersistence());
            // MQTT的连接设置
            options = new MqttConnectOptions();
            //-----------security begin--------------
            SSLSocketFactory factory = SSLUtil.getSSLSocketFactory(sslPemPath + "rootCA.crt",sslPemPath +"chainca3.crt",sslPemPath + "chainca3.key","shihucx");
            options.setSocketFactory(factory);
            //-----------end of security ------------
            // 设置是否清空session,这里如果设置为false表示服务器会保留客户端的连接记录，设置为true表示每次连接到服务器都以新的身份连接
            options.setCleanSession(false);
            // 设置连接的用户名
            options.setUserName(userName);
            // 设置连接的密码
            options.setPassword(passWord.toCharArray());
            // 设置超时时间 单位为秒
            options.setConnectionTimeout();
            // 设置会话心跳时间 单位为秒 服务器会每隔1.5*20秒的时间向客户端发送个消息判断客户端是否在线，但这个方法并没有重连的机制
            options.setKeepAliveInterval();
            // 设置重连机制
            options.setAutomaticReconnect(true);
            // 设置回调
            client.setCallback(new PushCallback());
            MqttTopic topic = client.getTopic(TOPIC1);
            //setWill方法，如果项目中需要知道客户端是否掉线可以调用该方法。设置最终端口的通知消息
            //options.setWill(topic, "close".getBytes(), 2, true);//遗嘱
            client.connect(options);
            //订阅消息
            int[] Qos  = {};
            String[] topic1 = {TOPIC1};
            client.subscribe(topic1, Qos);

        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    public static void main(String[] args) throws MqttException {
        //System.setProperty("javax.net.debug", "ssl,handshake");
        SSLChainConsumer client = new SSLChainConsumer();
        client.start();
    }
}

package com.taikang.iot.re.security;

import org.apache.log4j.Logger;
import org.bouncycastle.asn1.pkcs.RSAPrivateKey;
import org.bouncycastle.util.io.pem.PemObject;
import org.bouncycastle.util.io.pem.PemReader;

import javax.net.ssl.*;
import java.io.*;
import java.security.*;
import java.security.cert.Certificate;
import java.security.cert.CertificateFactory;
import java.security.cert.X509Certificate;
import java.security.spec.InvalidKeySpecException;
import java.security.spec.PKCS8EncodedKeySpec;
import java.security.spec.RSAPrivateKeySpec;

/**
 * @Author: chengsh05
 * @Date: 2019/3/1 17:51
 */
public class SSLUtil {

    private static Logger logger = Logger.getLogger(OpensslHelper.class);

    /**
     * 利用开源的工具类解析openssl私钥，openssl私钥文件格式为pem，需要去除页眉页脚后才能被java读取
     *
     * @param file
     * @return
     */
    public static PrivateKey getPrivateKey(File file) {
        if (file == null) {
            return null;
        }
        PrivateKey privKey = null;
        PemReader pemReader = null;
        try {
            pemReader = new PemReader(new FileReader(file));
            PemObject pemObject = pemReader.readPemObject();
            byte[] pemContent = pemObject.getContent();
            //支持从PKCS#1或PKCS#8 格式的私钥文件中提取私钥
            if (pemObject.getType().endsWith("RSA PRIVATE KEY")) {
                /*
                 * 取得私钥  for PKCS#1
                 * openssl genrsa 默认生成的私钥就是PKCS1的编码
                 */
                RSAPrivateKey asn1PrivKey = RSAPrivateKey.getInstance(pemContent);
                RSAPrivateKeySpec rsaPrivKeySpec = new RSAPrivateKeySpec(asn1PrivKey.getModulus(), asn1PrivKey.getPrivateExponent());
                KeyFactory keyFactory= KeyFactory.getInstance("rsa");
                privKey= keyFactory.generatePrivate(rsaPrivKeySpec);
            } else if (pemObject.getType().endsWith("PRIVATE KEY")) {
                /*
                 * 通过openssl pkcs8 -topk8转换为pkcs8，例如（-nocrypt不做额外加密操作）：
                 * openssl pkcs8 -topk8 -in pri.key -out pri8.key -nocrypt
                 *
                 * 取得私钥 for PKCS#8
                 */
                PKCS8EncodedKeySpec privKeySpec = new PKCS8EncodedKeySpec(pemContent);
                KeyFactory kf = KeyFactory.getInstance("rsa");
                privKey = kf.generatePrivate(privKeySpec);
            }
        } catch (FileNotFoundException e) {
            logger.error("read private key fail,the reason is the file not exist");
            e.printStackTrace();
        } catch (IOException e) {
            logger.error("read private key fail,the reason is :"+e.getMessage());
            e.printStackTrace();
        } catch (NoSuchAlgorithmException e) {
            logger.error("read private key fail,the reason is :"+e.getMessage());
            e.printStackTrace();
        } catch (InvalidKeySpecException e) {
            logger.error("read private key fail,the reason is :"+e.getMessage());
            e.printStackTrace();
        }  finally {
            try {
                if (pemReader != null) {
                    pemReader.close();
                }
            } catch (IOException e) {
                logger.error(e.getMessage());
            }
        }
        return privKey;
    }

    /**
     * 获取SSLContext，基于CA， Certificate， key及密码进行SSL上下文的创建
     *
     * @param caPath
     * @param crtPath
     * @param keyPath
     * @param password
     * @return
     * @throws Exception
     */
    private static SSLContext getSSLContext(String caPath, String crtPath, String keyPath, String password) throws Exception {
        /*
         * CA证书是用来认证服务端的，这里的CA就是一个公认的认证证书
         * TrustManagerFactory 管理的是授信的CA证书，所以KeyStore里面存放的不需要私钥信息，通常也不可能有
         */
        CertificateFactory cAf = CertificateFactory.getInstance("X.509");
        FileInputStream caIn = new FileInputStream(caPath);
        X509Certificate ca = (X509Certificate) cAf.generateCertificate(caIn);
        KeyStore caKs = KeyStore.getInstance("JKS");
        caKs.load(null, password.toCharArray());
        caKs.setCertificateEntry("ca1", ca); //可以通过设置alias不同，配置多个ca实例，即配置多个可信的root CA。
        TrustManagerFactory tmf = TrustManagerFactory.getInstance("PKIX");
        tmf.init(caKs);
        caIn.close();

        //这个客户端证书，是用来发送给服务端的，准备做双向验证用的。
        CertificateFactory cf = CertificateFactory.getInstance("X.509");
        FileInputStream crtIn = new FileInputStream(crtPath);
        X509Certificate caCert = (X509Certificate) cf.generateCertificate(crtIn);
        crtIn.close();

        //客户端私钥，是用来处理双向SSL验证中服务端用客户端证书加密的数据的解密（解析签名）工具
        KeyStore ks = KeyStore.getInstance("JKS");
        ks.load(null, password.toCharArray());
        ks.setCertificateEntry("certificate3", caCert);
        String sslPath = "E:\\HOWTO\\emqtt-ssl\\self1\\";
        FileInputStream crtIn1 = new FileInputStream(sslPath + "chainca1.crt");
        FileInputStream crtIn2 = new FileInputStream(sslPath + "chainca2.crt");
        X509Certificate caCert1 = (X509Certificate) cf.generateCertificate(crtIn1);
        X509Certificate caCert2 = (X509Certificate) cf.generateCertificate(crtIn2);
        crtIn1.close();
        crtIn2.close();
        ks.setCertificateEntry("certificate1", caCert1);
        ks.setCertificateEntry("certificate2", caCert2);

        PrivateKey privateKey = getPrivateKey(new File(keyPath));
        /*
         * 注意：下面这行代码中非常重要的一点是：
         * setKeyEntry这个函数的第二个参数 password，他不是指私钥的加密密码，只是KeyStore对这个私钥进行管理设置的密码
         *
         * setKeyEntry中最后一个参数，chain的顺序是证书链中越靠近当前privateKey节点的证书，越靠近数字下标0的位置。即chain[0]是privateKey对应的证书，
         * chain[1]是签发chain[0]的证书，以此类推，有chain[i+1]签发chain[i]的关系。
         */
        ks.setKeyEntry("private-key", privateKey, password.toCharArray(), new Certificate[]{caCert, caCert2, caCert1});

        /*
         * KeyManagerFactory必须是证书和私钥配对使用，即KeyStore里面装载客户端证书以及对应的私钥，双向SSL验证需要。
         */
        KeyManagerFactory kmf = KeyManagerFactory.getInstance("PKIX");
        kmf.init(ks, password.toCharArray());

        /*
         * 最后创建SSL套接字工厂 SSLSocketFactory
         * 注意：这里，SSLContext不支持TLSv2创建
         */
        SSLContext context = SSLContext.getInstance("TLSv1");
        KeyManager[] kms = kmf.getKeyManagers();
        TrustManager[] tms = tmf.getTrustManagers();
        context.init(kms, tms, new SecureRandom());
        return context;
    }

    /**
     * 基于给定的CA文件，客户端证书文件以及客户端私钥文件，进行SSL上下文环境的构建， 此处创建的SSLSocketFactory是支持双向SSL验证的。
     *
     * NOTE： 证书及秘钥文件，都是通过openssl创建获取的。
     *
     * @param caPath CA证书文件
     * @param crtPath 客户证书文件
     * @param keyPath 客户私钥文件
     * @param password KeyStore存储私钥配置的安全密码，类似数据库存了数据，想访问，需要密码一样。
     * @return
     * @throws Exception
     */
    public static SSLSocketFactory getSSLSocketFactory(String caPath, String crtPath, String keyPath, String password) throws Exception {
        SSLContext ctx = getSSLContext(caPath, crtPath, keyPath, password);
        SSLSocketFactory factory = ctx.getSocketFactory();
        return factory;
    }
}

package com.taikang.iot.re.demo;

import org.eclipse.paho.client.mqttv3.IMqttDeliveryToken;
import org.eclipse.paho.client.mqttv3.MqttCallback;
import org.eclipse.paho.client.mqttv3.MqttMessage;

/**
 * @Author: chengsh05
 * @Date: 2019/1/15 10:02
 */
public class PushCallback implements MqttCallback {

    public void connectionLost(Throwable cause) {
        // 连接丢失后，一般在这里面进行重连
        System.out.println("连接断开，可以做重连");
    }

    public void deliveryComplete(IMqttDeliveryToken token) {
        System.out.println("deliveryComplete---------" + token.isComplete());
    }

    public void messageArrived(String topic, MqttMessage message) throws Exception {
        // subscribe后得到的消息会执行到这里面
        System.out.println("Qos : " + message.getQos() + ", Topic :" + topic);
        System.out.println("Sub : " + new String(message.getPayload()) + "\r\n");
    }
}

这里我要重点说明的是KeyStore里面的函数setKeyEntry,下面是JDK8的api说明：

KeyStore的配置用来做双向验证的关键部分：
public final void setKeyEntry(String alias,
                              Key key,
                              char[] password,
                              Certificate[] chain)
                       throws KeyStoreException将给定的密钥分配给给定的别名，并使用给定的密码进行保护。
如果给定的密钥是类型java.security.PrivateKey ，它必须附有一个证书链，证明相应的公钥。 

如果给定的别名已经存在，与它相关联的密钥库信息将被给定的密钥（也可能是证书链）覆盖。 

参数
alias - 别名
key - 与别名 key的关键
password - 密码保护密钥
chain - 相应公钥的证书链（仅当给定键为 java.security.PrivateKey类型 java.security.PrivateKey ）。 

KeyStore中的注意事项：
1. setKeyEntry这个函数的第二个参数 password，他不是指私钥的加密密码，只是KeyStore对这个私钥进行管理设置的密码
2. setKeyEntry中最后一个参数，chain的顺序是证书链中越靠近当前privateKey节点的证书，越靠近数字下标0的位置。即chain[0]是privateKey对应的证书，
chain[1]是签发chain[0]的证书，以此类推，有chain[i+1]签发chain[i]的关系

正确的配置方式如下：
ks.setKeyEntry("private-key", privateKey, password.toCharArray(), new Certificate[]{caCert,caCert2,caCert1});

正确配置下，paho客户端和emqtt服务端消息交互的wireshark的截图如下：

对应setKeyEntry中的第三个参数，certificate类型的数组，证书链的关系可以涵盖rootCA，也可以不涵盖rootCA，但是，从当前证书到根证书之间的中间证书必须要在这个证书链中，且顺序必须正确，否则会出现：

1）证书链节点不全，即rootCA之前的节点有缺失，会出现服务端认证身份识别

ks.setKeyEntry("private-key", privateKey, password.toCharArray(), new Certificate[]{caCert, caCert2});

错误信息如下：

Warning: no suitable certificate found - continuing without client authentication
*** Certificate chain
<Empty>

ssl: Ignoring alias private-key: issuers do not match
ssl: KeyMgr: no matching key found

。。。。。。。。。。。。。。。。。。。。。。。。。


MqttException () - javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure
    at org.eclipse.paho.client.mqttv3.internal.ExceptionHelper.createMqttException(ExceptionHelper.java:)
    at org.eclipse.paho.client.mqttv3.internal.ClientComms$ConnectBG.run(ClientComms.java:)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:)
    at java.util.concurrent.FutureTask.run(FutureTask.java:)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$(ScheduledThreadPoolExecutor.java:)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:)
    at java.lang.Thread.run(Thread.java:)
Caused by: javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure
    at sun.security.ssl.Alerts.getSSLException(Alerts.java:)
    at sun.security.ssl.Alerts.getSSLException(Alerts.java:)
    at sun.security.ssl.SSLSocketImpl.recvAlert(SSLSocketImpl.java:)
    at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:)
    at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:)
    at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:)
    at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:)
    at org.eclipse.paho.client.mqttv3.internal.SSLNetworkModule.start(SSLNetworkModule.java:)
    at org.eclipse.paho.client.mqttv3.internal.ClientComms$ConnectBG.run(ClientComms.java:)
    ...  more

2）证书链节点是全的，但是顺序不对

ks.setKeyEntry("private-key", privateKey, password.toCharArray(), new Certificate[]{caCert1, caCert2, caCert});

MqttException () - javax.net.ssl.SSLHandshakeException: Received fatal alert: unknown_ca
    at org.eclipse.paho.client.mqttv3.internal.ExceptionHelper.createMqttException(ExceptionHelper.java:)
    at org.eclipse.paho.client.mqttv3.internal.ClientComms$ConnectBG.run(ClientComms.java:)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:)
    at java.util.concurrent.FutureTask.run(FutureTask.java:)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$(ScheduledThreadPoolExecutor.java:)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:)
    at java.lang.Thread.run(Thread.java:)
Caused by: javax.net.ssl.SSLHandshakeException: Received fatal alert: unknown_ca
    at sun.security.ssl.Alerts.getSSLException(Alerts.java:)
    at sun.security.ssl.Alerts.getSSLException(Alerts.java:)
    at sun.security.ssl.SSLSocketImpl.recvAlert(SSLSocketImpl.java:)
    at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:)
    at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:)
    at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:)
    at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:)
    at org.eclipse.paho.client.mqttv3.internal.SSLNetworkModule.start(SSLNetworkModule.java:)
    at org.eclipse.paho.client.mqttv3.internal.ClientComms$ConnectBG.run(ClientComms.java:)
    ...  more

此博文到此，有需要探讨的，欢迎关注我的博客，共同进步，安全非小事，点滴积累吧