PySpider HTTP 599: SSL certificate problem错误的解决方法

在用 PySpider 爬取 https 开头的网站的时候遇到了 HTTP 599: SSL certificate problem: self signed certificate in certificate chain 的错误。

经过一番排查，解决方案总结如下

错误原因

这个错误会发生在请求 https 开头的网址，SSL 验证错误，证书有误。

报错如下：

[E 180823 09:18:21 base_handler:203] HTTP 599: SSL certificate problem: self signed certificate in certificate chain
    Traceback (most recent call last):
      File "f:\python\python36\lib\site-packages\pyspider\libs\base_handler.py", line 196, in run_task
        result = self._run_task(task, response)
      File "f:\python\python36\lib\site-packages\pyspider\libs\base_handler.py", line 175, in _run_task
        response.raise_for_status()
      File "f:\python\python36\lib\site-packages\pyspider\libs\response.py", line 172, in raise_for_status
        six.reraise(Exception, Exception(self.error), Traceback.from_string(self.traceback).as_traceback())
      File "f:\python\python36\lib\site-packages\six.py", line 692, in reraise
        raise value.with_traceback(tb)
      File "f:\python\python36\lib\site-packages\pyspider\fetcher\tornado_fetcher.py", line 378, in http_fetch
        response = yield gen.maybe_future(self.http_client.fetch(request))
      File "f:\python\python36\lib\site-packages\tornado\httpclient.py", line 102, in fetch
        self._async_client.fetch, request, **kwargs))
      File "f:\python\python36\lib\site-packages\tornado\ioloop.py", line 458, in run_sync
        return future_cell[0].result()
      File "f:\python\python36\lib\site-packages\tornado\concurrent.py", line 238, in result
        raise_exc_info(self._exc_info)
      File "<string>", line 4, in raise_exc_info
    Exception: HTTP 599: SSL certificate problem: self signed certificate in certificate chain

最简单的解决方法是：

在 crawl 方法中加入忽略证书验证的参数，validate_cert=False，即

 

 

1	self.crawl(url,callback=method_name,validate_cert=False)