抓取https网页时,报错sun.security.validator.ValidatorException: PKIX path building failed 解决办法

原因是https证书问题,java抓取时忽略掉证书才能访问。jsoup在调用前先执行下以下忽略证书请求就可以了。

try {
//先调用下忽略https证书的再请求才可以
HttpsUrlValidator.retrieveResponseFromServer(url);
doc = Jsoup
.connect(url)
.header("User-Agent",rand_agents)
.timeout(10000).get();
body = doc.getElementsByTag("body").html();
} catch (Exception e) {
log.info(e.getMessage());
}

HttpsUrlValidator 类:

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL; import javax.net.ssl.HostnameVerifier;
import javax.net.ssl.HttpsURLConnection;
import javax.net.ssl.SSLSession; public class HttpsUrlValidator { static HostnameVerifier hv = new HostnameVerifier() {
public boolean verify(String urlHostName, SSLSession session) {
System.out.println("Warning: URL Host: " + urlHostName + " vs. "
+ session.getPeerHost());
return true;
}
}; public final static String retrieveResponseFromServer(final String url) {
HttpURLConnection connection = null; try {
URL validationUrl = new URL(url);
trustAllHttpsCertificates();
HttpsURLConnection.setDefaultHostnameVerifier(hv); connection = (HttpURLConnection) validationUrl.openConnection();
final BufferedReader in = new BufferedReader(new InputStreamReader(
connection.getInputStream())); String line;
final StringBuffer stringBuffer = new StringBuffer(255); synchronized (stringBuffer) {
while ((line = in.readLine()) != null) {
stringBuffer.append(line);
stringBuffer.append("\n");
}
return stringBuffer.toString();
} } catch (final IOException e) {
System.out.println(e.getMessage());
return null;
} catch (final Exception e1){
System.out.println(e1.getMessage());
return null;
}finally {
if (connection != null) {
connection.disconnect();
}
}
} public static void trustAllHttpsCertificates() throws Exception {
javax.net.ssl.TrustManager[] trustAllCerts = new javax.net.ssl.TrustManager[1];
javax.net.ssl.TrustManager tm = new miTM();
trustAllCerts[0] = tm;
javax.net.ssl.SSLContext sc = javax.net.ssl.SSLContext
.getInstance("SSL");
sc.init(null, trustAllCerts, null);
javax.net.ssl.HttpsURLConnection.setDefaultSSLSocketFactory(sc
.getSocketFactory());
} static class miTM implements javax.net.ssl.TrustManager,
javax.net.ssl.X509TrustManager {
public java.security.cert.X509Certificate[] getAcceptedIssuers() {
return null;
} public boolean isServerTrusted(
java.security.cert.X509Certificate[] certs) {
return true;
} public boolean isClientTrusted(
java.security.cert.X509Certificate[] certs) {
return true;
} public void checkServerTrusted(
java.security.cert.X509Certificate[] certs, String authType)
throws java.security.cert.CertificateException {
return;
} public void checkClientTrusted(
java.security.cert.X509Certificate[] certs, String authType)
throws java.security.cert.CertificateException {
return;
}
} }

抓取https网页时,报错sun.security.validator.ValidatorException: PKIX path building failed 解决办法的更多相关文章

  1. mvn 编译报错mavn sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested targ

    mavn 编译报错: mavn sun.security.validator.ValidatorException: PKIX path building failed: sun.security.p ...

  2. Maven:sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

    还是记录使用 maven 时遇到的问题. 一.maven报错 maven package 进行打包时出现了以下报错: Non-resolvable parent POM for com.wpbxin: ...

  3. 异常信息:javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed

    上周五遇到一个问题,工程本地编译运行正常,打包本地tomcat运行也正常.部署到测试环境报错: 2017-05-05 09:38:11.645 ERROR [HttpPoolClientsUtil.j ...

  4. 解决 sun.security.validator.ValidatorException: PKIX path building failed

    今天用java HttpClients写爬虫在访问某Https站点报如下错误: sun.security.validator.ValidatorException: PKIX path buildin ...

  5. javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certificatio

    场景:Java调用PHP接口,代码部署在服务器上后,调用报错,显示PHP服务器那边证书我这边服务器不信任(我猜的). 异常信息: 2019-08-06 14:00:09,102 [http-nio-4 ...

  6. javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed

    1.使用HttpClient4.3 调用https出现如下错误: javax.net.ssl.SSLHandshakeException: sun.security.validator.Validat ...

  7. sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

    httpclient-4.5.jar 定时发送http包,忽然有一天报错,http证书变更引起的. 之前的代码 try { CloseableHttpClient httpClient = build ...

  8. ES访问遇到sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

    cmd命令cd到jre/bin目录下 输入命令keytool -import -alias 别名 -keystore cacerts -file ‪C://certs//elasticsearch// ...

  9. javax.net.ssl.sslhandshakeException:sun.security.validator.validatorException:PKIX path buildind failed

    前段时间开发的一个需求,需要通过图片URL获取图片的base64编码,测试的时候使用的是百度图片的url,测试没有问题,但是发布后测试时报如下错: javax.net.ssl.sslhandshake ...

随机推荐

  1. C++的类大小

    有以下C++类 #pragma pack(push) #pragma pack(1) class task /*:public OSThread*/{ //public: // void Entry( ...

  2. 帝国cms建站-动态获取栏目id

    <?php $fcr=explode('|',$class_r[$GLOBALS[navclassid]][featherclass]); $topbclassid=$fcr[1]?$fcr[1 ...

  3. Linux查看某个端口使用情况并kill

    Linux查看某个端口使用情况并kill 例如查看8083端口的状态: netstat -apn | grep 8083 tcp 0 0 192.168.2.17:8083 0.0.0.0:* LIS ...

  4. Mongodb 基础 数据导入导出和用户管理

    数据导出 导出命令使用方式 mongoexport---->csv/json使用 参数: -d 库名 -c 表名 -f field1,field2....列名 -q 查询条件 -o 导出文件名 ...

  5. 新手详解JAVA+数据库+JSP完成简单页面

    本篇以数据库添加为例(本例中数据库名为“xinxi”表单名字为“stud”) 准备---实体层: package entity; public class Student { private Stri ...

  6. node使用 log4js

    log4js //配置日志的输出级别,共ALL<TRACE<DEBUG<INFO<WARN<ERROR<FATAL<MARK<OFF八个级别,defau ...

  7. 20165321 学习基础与C语言学习心得

    一.技能学习 我其实在小时候学过挺多东西,在我小学的时候,我曾经短时间地学过小提琴.拉丁舞.国画.书法,但是,由于各种原因,都没有继续学习下去.后来,在我小学四年级的时候,我接触到了二胡,于是,我开始 ...

  8. 原来CNN是这样提取图像特征的。。。

    对于即将到来的人工智能时代,作为一个有理想有追求的程序员,不懂深度学习(Deep Learning)这个超热的领域,会不会感觉马上就out了?作为机器学习的一个分支,深度学习同样需要计算机获得强大的学 ...

  9. cocos2d JS 创建实现换行功能的聊天文本 testLable

    -- 创建实现换行功能的文本 function LoadLabelHeroIntroduce(objLabelContent)  //传入文本 ---------------------------- ...

  10. cocos2d JS 监听键盘触摸响应事件(cc.EventListener.KEYBOARD)

    除了可以监听键盘按键,还可以是终端设备的各个菜单键,都能使用同一个监听器来进行处理. //给statusLabel绑定键盘事件 cc.eventManager.addListener({ event: ...