爬虫四部曲，本人按自己的步骤来写，可能有很多漏洞，望各位大神指点指点

1、创建项目

scrapy startproject xiaohuawang

scrapy.cfg: 项目的配置文件
xiaohuawang/: 该项目的python模块。之后您将在此加入代码。
xiaohuawang/items.py: 项目中的item文件.
xiaohuawang/pipelines.py: 项目中的pipelines文件.
xiaohuawang/settings.py: 项目的设置文件.

2、进入xiaohuawang目录然创建爬虫名及爬取范围

cd xiaohuawang

scrapy genspider xhwang xiaohuar.com

3、创建爬取的项目（xiaohuawang/item.py）

4、编写爬虫

xiaohuawang/spider/xhwang.py: 爬虫文件

5、保存数据（此处使用try...expect是为剔除完整的图片url）

6、打开数据保存开关（setting.py）如果有第五步的保存数据的必要,此处需要打开

7、开始爬取之路

在终端键入scrapy crawl xhwang 本地目录出现如下数据（截取部分）校花还看得过去，这校草嘛除了李易峰其他都没见过o(*￣︶￣*)o

终端出现的界面

2018-10-15 09:52:25 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/7e191d68e7eb029b4c1da1a3808e5059.jpg HTTP/1.1" 200 53517
2018-10-15 09:52:26 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.comhttp:80
jpg_url is error!!!
2018-10-15 09:52:26 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.comhttp:80
jpg_url is error!!!
2018-10-15 09:52:26 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:26 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/63d5b955486e38d0fb664bcc9e8d96d6.jpg HTTP/1.1" 200 32479
2018-10-15 09:52:26 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:26 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/20160109/1b82558a72a175802de1c041d7c60813.jpg HTTP/1.1" 200 779046
2018-10-15 09:52:31 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:31 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/20170608/f07501a9d1fe9945e2fb96f0890a8d23.jpg HTTP/1.1" 200 113462
2018-10-15 09:52:32 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:32 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/20151018/e03c74b0e1fe13caab5e138c8b013a00.jpg HTTP/1.1" 200 33869
2018-10-15 09:52:32 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:32 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/9c9e3de9965854e46404af656d86a596.jpg HTTP/1.1" 200 212437
2018-10-15 09:52:34 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:35 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/180584665e90af45905e3ab2ab0bdb62.jpg HTTP/1.1" 200 32805
2018-10-15 09:52:35 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:35 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/f96fe78647d53691e5cb3d9f483d950e.jpg HTTP/1.1" 200 161345
2018-10-15 09:52:37 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.comhttp:80
jpg_url is error!!!
2018-10-15 09:52:37 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:37 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/1b3b53eeae45059d3f46490d3eb45672.jpg HTTP/1.1" 200 71261
2018-10-15 09:52:37 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:38 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/8cb2bf048b0fdcbb8aabe6ca599eaa2b.jpg HTTP/1.1" 200 78371
2018-10-15 09:52:38 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:38 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/d02e95d32d1aed5fb46d5744ed5f9729.jpg HTTP/1.1" 200 14788
2018-10-15 09:52:38 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:38 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/65d5cfc0d5f54559c07073edf2421da7.jpg HTTP/1.1" 200 836584
2018-10-15 09:52:43 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:43 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/901b797f3ed0c5e1c7e861496ebe1c2d.jpg HTTP/1.1" 200 75831
2018-10-15 09:52:43 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:43 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/3047a6706453b987c26f54beba3e32ae.jpg HTTP/1.1" 200 81777
2018-10-15 09:52:44 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:44 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/20151003/9177cd57017a50783a94a4c700bae808.jpg HTTP/1.1" 200 194601
2018-10-15 09:52:46 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:46 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/b332da0c2a45f5ec08add990255b0071.jpg HTTP/1.1" 200 31974
2018-10-15 09:52:46 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.comhttp:80
jpg_url is error!!!
2018-10-15 09:52:46 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.comhttp:80
jpg_url is error!!!
2018-10-15 09:52:46 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:47 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/0540ae30cfbdf80a8121d5d6e14e8b62.jpg HTTP/1.1" 200 80261
2018-10-15 09:52:48 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:48 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/7e9a5cf7fd86a4afd8da51afe24cf82d.jpg HTTP/1.1" 200 48760
2018-10-15 09:52:48 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:48 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/ef1f9ff054246626f7a3edbcaab11810.jpg HTTP/1.1" 200 85468
2018-10-15 09:52:49 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:49 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/67af1100e5a0ba7e0de35f06e6d3c5fe.jpg HTTP/1.1" 200 46750
2018-10-15 09:52:49 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:50 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/4e0406f4073c4120b6e93cb08eafaf76.jpg HTTP/1.1" 200 11429
2018-10-15 09:52:50 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:50 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/20150915/f51d151afdd59c2f0b94ae5ebd9df2e5.jpg HTTP/1.1" 200 187448
2018-10-15 09:52:51 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:51 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/974244ea878d9d58cdeb5e581ba00c2d.png HTTP/1.1" 200 51702
2018-10-15 09:52:51 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:52 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/4bbf1aebc9e0ff735539e47185f07d86.jpg HTTP/1.1" 200 55832
2018-10-15 09:52:52 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:52 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/79f175dd8866f8bd756d0d663b56f63f.jpg HTTP/1.1" 200 97644
2018-10-15 09:52:53 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:53 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/20160128/edb010401a9314f51ba85dcef5628640.jpg HTTP/1.1" 200 51555
2018-10-15 09:52:53 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:53 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/38c8daefae96096777168b878ac12457.jpg HTTP/1.1" 200 48988
2018-10-15 09:52:54 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:54 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/f57c0b3d74190fcdf3496f29c495a9a8.jpg HTTP/1.1" 200 89019
2018-10-15 09:52:55 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:55 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/20161209/b0629634f424d3fc92ccf5e3311cf26d.jpg HTTP/1.1" 200 233679
2018-10-15 09:52:56 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:56 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/0d1c02b6d4eb2d49e06b85c920226e12.jpg HTTP/1.1" 200 35265
2018-10-15 09:52:56 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:57 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/20150917/1ea043c66124620e7bef39abfeaa7309.jpg HTTP/1.1" 200 102374
2018-10-15 09:52:59 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:59 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/ba9f9d31a9c1a74c0992fc53642351b4.jpg HTTP/1.1" 200 48760
2018-10-15 09:53:00 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:00 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/42d1ff3f5ee54c737a2a20691a51dd2e.jpg HTTP/1.1" 200 117388
2018-10-15 09:53:01 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:01 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/20151004/ff3c6b4a769142bcf07d36a38090e9ca.jpg HTTP/1.1" 200 238026
2018-10-15 09:53:02 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:02 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/54859a36a815d4cc9e4bfd0fa226a801.jpg HTTP/1.1" 200 65610
2018-10-15 09:53:03 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:03 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/20150829/61d547839957d38499af8a030f181bbc.jpg HTTP/1.1" 200 34660
2018-10-15 09:53:06 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:06 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/20170210/8e6c33aec8013aee65eb677f64ead2ff.jpg HTTP/1.1" 200 144376
2018-10-15 09:53:07 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:07 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/54da0d784895352c52c8de4c4724702b.jpg HTTP/1.1" 200 70823
2018-10-15 09:53:07 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:08 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/20151229/f7235e51a566a9b25343a3a034f47291.jpg HTTP/1.1" 200 58763
2018-10-15 09:53:08 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:08 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/ed3640812048582f544140c1dc29c701.jpg HTTP/1.1" 200 69413
2018-10-15 09:53:10 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:10 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/72edbf7ff56c981b82effd1dd2a181ff.jpg HTTP/1.1" 200 101817
2018-10-15 09:53:13 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:13 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/9aad0c431d4508d2924364532e5fb405.jpg HTTP/1.1" 200 77562
2018-10-15 09:53:14 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:14 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/6cc7ad88b47e1191cb112deed98a96cd.jpg HTTP/1.1" 200 21454
2018-10-15 09:53:14 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:14 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/0cb7df1d7ee292ddf6a7de7c9084e97c.jpg HTTP/1.1" 200 96486
2018-10-15 09:53:15 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:15 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/20170221/a6e051fd0752d6ad3beca2014a1f620e.jpg HTTP/1.1" 200 31762
2018-10-15 09:53:15 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:16 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/895782554cccd62f3819c0f912d8afba.jpg HTTP/1.1" 200 60290
2018-10-15 09:53:16 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:16 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/1a645a5896cc1a2989c9778c0dcd3d4e.jpg HTTP/1.1" 200 36327
2018-10-15 09:53:17 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:17 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/c077e29c5f9b89eb404505567df17c0c.jpg HTTP/1.1" 200 20020
2018-10-15 09:53:17 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:17 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/f825758f36e1c2c346f4f8f957df5430.jpg HTTP/1.1" 200 39516
2018-10-15 09:53:17 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:17 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/782cccaab95f8bdbcd6937c67cbaa3b0.jpg HTTP/1.1" 200 380076
2018-10-15 09:53:22 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:22 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/97f350f3e0fe65a8f7c768664e1eaca5.jpg HTTP/1.1" 200 422140
2018-10-15 09:53:25 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:25 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/883ea459b664518f60d9ea6fd9f49f0d.jpg HTTP/1.1" 200 5545
2018-10-15 09:53:25 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:25 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/281fd6f85bf33ace9b99b02031cb2cb9.jpg HTTP/1.1" 200 55469
2018-10-15 09:53:26 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:26 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/46cfceb6eac4bd6d5ab051098f413bdf.jpg HTTP/1.1" 200 132728
2018-10-15 09:53:27 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.comhttp:80
jpg_url is error!!!
2018-10-15 09:53:27 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:27 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/c62ebcca53796107aaad2fe5577cde44.jpg HTTP/1.1" 200 10273
2018-10-15 09:53:27 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.comhttp:80
jpg_url is error!!!
2018-10-15 09:53:27 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:27 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/2219bc2968708eb47ca47b618c96e003.jpg HTTP/1.1" 200 128837
2018-10-15 09:53:28 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:28 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/50c4400bda96f2305fc74de79aea913b.jpg HTTP/1.1" 200 42574
2018-10-15 09:53:29 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:29 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/cfdffb8e41273fbc287293bd2d0a3e59.jpg HTTP/1.1" 200 40472
2018-10-15 09:53:30 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:30 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/20150905/ffa892119baef28dadfcadc3b7abd50a.jpg HTTP/1.1" 200 128271
2018-10-15 09:53:31 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:31 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/20170331/5164f1dc347b2cd5d6b57088d2614c4d.jpg HTTP/1.1" 200 30121
2018-10-15 09:53:31 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:31 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/299823f15c021ede81f844b30b8d543a.jpg HTTP/1.1" 200 76565
2018-10-15 09:53:32 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:32 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/bf464364888cfecaf95b0a2a9e3407bf.png HTTP/1.1" 200 305039
2018-10-15 09:53:36 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:36 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/a9843ed50a56e020ed78e40a348bcb5c.jpg HTTP/1.1" 200 87337
2018-10-15 09:53:37 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:37 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/f9fe702b5e01210af5afd970e8499090.jpg HTTP/1.1" 200 107942
2018-10-15 09:53:38 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:38 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/d3e3d2526686463b4a1406e039388ca6.jpg HTTP/1.1" 200 58479
2018-10-15 09:53:38 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:38 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/1e29bc94ee7ebde0ae50bc2b5bd112c9.jpg HTTP/1.1" 200 66940
2018-10-15 09:53:40 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:40 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/db4d5e02fdea2e910ee35eb6b4979a99.jpg HTTP/1.1" 200 7716
2018-10-15 09:53:40 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:41 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/afaf7502ffca5a0bb052eddba8195509.jpg HTTP/1.1" 200 47045
2018-10-15 09:53:41 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:41 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/90d63585d7230a48ddd9516dbf0dd6ab.jpg HTTP/1.1" 200 74944
2018-10-15 09:53:42 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:43 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/689fa17c65c8d0f049521383d43a4ab2.jpg HTTP/1.1" 200 49714
2018-10-15 09:53:43 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:43 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/20151217/b5108ea8a7a4920b2d7bf6e27c0b1101.jpg HTTP/1.1" 200 111850
2018-10-15 09:53:44 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:44 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/e81bf99a232cdeee60dc65418a8c2602.png HTTP/1.1" 200 46237
2018-10-15 09:53:45 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:45 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/04c92ce23ee91e9cd857955e4a60b46d.jpg HTTP/1.1" 200 994018
2018-10-15 09:54:00 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:54:00 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/9a3315019ae4b15e84e18770f91e7f28.jpg HTTP/1.1" 200 99168
2018-10-15 09:54:02 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:54:02 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/2d64c4f98c23b4177ee1ae9725354ea4.jpg HTTP/1.1" 200 258723
2018-10-15 09:54:07 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:54:07 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/0ec2c52c6954837c2b588bb61e7e2e15.jpg HTTP/1.1" 200 47123
2018-10-15 09:54:07 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:54:07 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/20160220/b1e09b5d13c74f444806086a1a8aedbe.jpg HTTP/1.1" 200 33438
2018-10-15 09:54:08 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:54:11 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/6854f7987613ceac3a21049efa2f7047.jpg HTTP/1.1" 200 45545
2018-10-15 09:54:14 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:54:14 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/873316b54c2bc3bcc3573971fda73286.png HTTP/1.1" 200 36981
2018-10-15 09:54:14 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:54:14 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/117e999a87ef237115071d64b692b134.jpg HTTP/1.1" 200 286144
2018-10-15 09:54:16 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:54:16 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/24191b9f48d9f0ee58baf69e13a18c78.jpg HTTP/1.1" 200 704182
2018-10-15 09:54:21 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:54:21 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/2568b473a4a93b0be1366800d9194799.jpg HTTP/1.1" 200 40740
2018-10-15 09:54:21 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:54:21 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/0a83bccd885cba88a11eb93c5f789035.jpg HTTP/1.1" 200 102835
2018-10-15 09:54:22 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:54:23 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/a55cc2a57a2af7fad3477d799215ad05.jpg HTTP/1.1" 200 63726
2018-10-15 09:54:23 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:54:24 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/cd38314fd9c336afe56ef8d61cb8416d.jpg HTTP/1.1" 200 53809
2018-10-15 09:54:24 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:54:24 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/c0a77ff668ed2bb6292ec81a4c4105c7.jpg HTTP/1.1" 200 49287
2018-10-15 09:54:24 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:54:25 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/2bb8f36b6f540360d081ea17dad0d1c6.jpg HTTP/1.1" 200 59997
2018-10-15 09:54:25 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:54:25 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/20151213/e6f03eb1bb25208f230338ee76963d73.jpg HTTP/1.1" 200 126816
2018-10-15 09:54:28 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:54:28 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/c6bfce8e88fbeeb0ec94ed740ea02657.jpg HTTP/1.1" 200 25190
2018-10-15 09:54:28 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:54:29 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/9732937ef6b527bc9374bec637f209b4.jpg HTTP/1.1" 200 99894
2018-10-15 09:54:32 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:54:32 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/e0f286f6c4a83c5458c99e98d692a383.png HTTP/1.1" 200 46237
2018-10-15 09:54:33 [scrapy.core.scraper] DEBUG: Scraped from <200 http://www.xiaohuar.com/xiaocao/>
{'name': ['大连海事大学校草牟福鹏',
'广西农业职业技术学院校草王超树',
'四川师范大学校草李易峰',
'南开中学校草王源',
'重庆八中校草王俊凯',
'清华大学校草林时完',
'北京市海淀外国语实验学校校草鹿晗',
'北京师范大学珠海分校校草吴凡',
'湖南重点中学湖南师大附中校草易烊千玺',
'中国文化大学校草柯震东',
'清华大学校草刘睿睿',
'北京大学校草苑子文苑子豪',
'清华大学校草李润东',
'辽宁石油化工大学顺华能源学院校草李佩凡',
'新疆警官高等专科学校校草杨晨',
'浙江大学校草王平',
'云南国土资源学院校草李莘瞳',
'吉林大学校草夏昕濛',
'中国人民大学校草龙马',
'湖南师范附中梅溪湖中学校草易烊千玺',
'华中科技大学校草杨逸韬',
'四川师范大学校草潘子剑',
'四川警察学院校草孟祥鑫',
'大连理工校草仪炎彬',
'深圳大学校草甘东柏',
'中国民航飞行学院校草木拉丁',
'西南大学校草温浩',
'兰州交通大学校草柴铎',
'北京现代音乐学院校草陈沫燃',
'长沙理工大学城南学院校草黄庭帅',
'福建师范大学校草李晨希',
'中南民族大学校草李春天',
'清华大学校草孟湛东',
'浙江警察学院校草张超贺',
'杭州师范大学校草俊杰',
'白城师范学院校草袭新',
'江苏省司法警官高等职业学校校草魏一川',
'集美大学校草廖家汉',
'华中师范大学校草邹祖樊',
'中国美院校草凤飞飞',
'山东服装职业学院校草徐义强',
'江西南昌安义二中校草校草黄凡斌',
'上海杉达学院校草沈祎玮',
'哈密石油高级中学校草王竞',
'辽宁水利学院校草小马',
'广西贵港屌丝中学校草苏泽辰',
'华南师范大学校草沈斌',
'河南理工校草范博',
'寒校草秋',
'西南财经大学天府学院校草周月笙',
'南昌大学校草邓文才',
'啊实打实的校草阿萨德',
'北京师范大学校草秋寒',
'铁岭市外事学院校草董立鹏',
'四川帅哥校草刘亦晨',
'长春汽车工业高等专科学校校草史培元',
'南昌航空大学校草郭美城',
'云南警官学院校草李庆?',
'安徽财经大学校草查道杰',
'四川师范大学校草刘亦晨',
'赣南师范学院校草王思豪',
'福建师范大学校草李毅刚',
'北京电影学院校草巴宇嘉',
'西南政法大学校草董浩洋',
'安徽大学校草张峰',
'廊坊东方大学城河北体育学院校草馮政',
'珠海大学校草蔡璇',
'江南高中校草朴朗',
'重庆南开中学高中部校草王源',
'长春工程学院校草邸平涛',
'北京八维大学校草李超',
'玉溪校草杨玉金',
'上海师范大学天华学院校草沈辛峄',
'校草校草白天明',
'技师学院校草刘家森',
'泉州二中校草安然',
'专门磕药大学校草丁三圆',
'哈尔滨工业大学威海校区校草孙嘉良',
'泰山学院校草甄文忠',
'江汉大学校草曹文星',
'长沙理工大学城南学院校草朱泽贤',
'石家庄大学校草赵云龙',
'河南建筑职业技术学院校草杜健康',
'西南大学校草王壮',
'复旦校草胡瑞杰',
'临汾一中校草李江灜',
'清华校草詹凯',
'长沙理工大学城南学院校草毛创',
'东乡一中校草周凯波',
'鄱阳一中校草黄清',
'西南政法大学校草李帆',
'上杭职专校草曹永强',
'电子科技大学校草杨星',
'海南校草符是雄',
'曲阜远东职业技术学院校草王帅',
'西南交通大学校草路超',
'淮北师范大学校草刘家威',
'河北政法学院校草柏宇',
'鄱阳一中校草卢政辉',
'长沙理工大学城南学院校草朱泽贤'],
'url': ['/d/file/20150909/c76fa0a8c4d63f749fa3a6883bbdd4a8.jpg',
'/d/file/20161015/75541204516ea9e2118248fb374f706a.jpg',
'/d/file/7e191d68e7eb029b4c1da1a3808e5059.jpg',
'http://www.xiaohuar.com/d/file/20150512105953141.jpg',
'http://www.xiaohuar.com/d/file/20150813180033144.jpg',
'/d/file/63d5b955486e38d0fb664bcc9e8d96d6.jpg',
'/d/file/20160109/1b82558a72a175802de1c041d7c60813.jpg',
'/d/file/20170608/f07501a9d1fe9945e2fb96f0890a8d23.jpg',
'/d/file/20151018/e03c74b0e1fe13caab5e138c8b013a00.jpg',
'/d/file/9c9e3de9965854e46404af656d86a596.jpg',
'/d/file/180584665e90af45905e3ab2ab0bdb62.jpg',
'/d/file/f96fe78647d53691e5cb3d9f483d950e.jpg',
'http://www.xiaohuar.com/d/file/20140810073313127.jpg',
'/d/file/1b3b53eeae45059d3f46490d3eb45672.jpg',
'/d/file/8cb2bf048b0fdcbb8aabe6ca599eaa2b.jpg',
'/d/file/d02e95d32d1aed5fb46d5744ed5f9729.jpg',
'/d/file/65d5cfc0d5f54559c07073edf2421da7.jpg',
'/d/file/901b797f3ed0c5e1c7e861496ebe1c2d.jpg',
'/d/file/3047a6706453b987c26f54beba3e32ae.jpg',
'/d/file/20151003/9177cd57017a50783a94a4c700bae808.jpg',
'/d/file/b332da0c2a45f5ec08add990255b0071.jpg',
'http://www.xiaohuar.com/d/file/20150221160551154.jpg',
'http://www.xiaohuar.com/d/file/20150212230330155.jpg',
'/d/file/0540ae30cfbdf80a8121d5d6e14e8b62.jpg',
'/d/file/7e9a5cf7fd86a4afd8da51afe24cf82d.jpg',
'/d/file/ef1f9ff054246626f7a3edbcaab11810.jpg',
'/d/file/67af1100e5a0ba7e0de35f06e6d3c5fe.jpg',
'/d/file/4e0406f4073c4120b6e93cb08eafaf76.jpg',
'/d/file/20150915/f51d151afdd59c2f0b94ae5ebd9df2e5.jpg',
'/d/file/974244ea878d9d58cdeb5e581ba00c2d.png',
'/d/file/4bbf1aebc9e0ff735539e47185f07d86.jpg',
'/d/file/79f175dd8866f8bd756d0d663b56f63f.jpg',
'/d/file/20160128/edb010401a9314f51ba85dcef5628640.jpg',
'/d/file/38c8daefae96096777168b878ac12457.jpg',
'/d/file/f57c0b3d74190fcdf3496f29c495a9a8.jpg',
'/d/file/20161209/b0629634f424d3fc92ccf5e3311cf26d.jpg',
'/d/file/0d1c02b6d4eb2d49e06b85c920226e12.jpg',
'/d/file/20150917/1ea043c66124620e7bef39abfeaa7309.jpg',
'/d/file/ba9f9d31a9c1a74c0992fc53642351b4.jpg',
'/d/file/42d1ff3f5ee54c737a2a20691a51dd2e.jpg',
'/d/file/20151004/ff3c6b4a769142bcf07d36a38090e9ca.jpg',
'/d/file/54859a36a815d4cc9e4bfd0fa226a801.jpg',
'/d/file/20150829/61d547839957d38499af8a030f181bbc.jpg',
'/d/file/20170210/8e6c33aec8013aee65eb677f64ead2ff.jpg',
'/d/file/54da0d784895352c52c8de4c4724702b.jpg',
'/d/file/20151229/f7235e51a566a9b25343a3a034f47291.jpg',
'/d/file/ed3640812048582f544140c1dc29c701.jpg',
'/d/file/72edbf7ff56c981b82effd1dd2a181ff.jpg',
'/d/file/9aad0c431d4508d2924364532e5fb405.jpg',
'/d/file/6cc7ad88b47e1191cb112deed98a96cd.jpg',
'/d/file/0cb7df1d7ee292ddf6a7de7c9084e97c.jpg',
'/d/file/20170221/a6e051fd0752d6ad3beca2014a1f620e.jpg',
'/d/file/895782554cccd62f3819c0f912d8afba.jpg',
'/d/file/1a645a5896cc1a2989c9778c0dcd3d4e.jpg',
'/d/file/c077e29c5f9b89eb404505567df17c0c.jpg',
'/d/file/f825758f36e1c2c346f4f8f957df5430.jpg',
'/d/file/782cccaab95f8bdbcd6937c67cbaa3b0.jpg',
'/d/file/97f350f3e0fe65a8f7c768664e1eaca5.jpg',
'/d/file/883ea459b664518f60d9ea6fd9f49f0d.jpg',
'/d/file/281fd6f85bf33ace9b99b02031cb2cb9.jpg',
'/d/file/46cfceb6eac4bd6d5ab051098f413bdf.jpg',
'http://www.xiaohuar.com/d/file/20150221155450109.jpg',
'/d/file/c62ebcca53796107aaad2fe5577cde44.jpg',
'http://www.xiaohuar.com/d/file/20150505170150104.jpg',
'/d/file/2219bc2968708eb47ca47b618c96e003.jpg',
'/d/file/50c4400bda96f2305fc74de79aea913b.jpg',
'/d/file/cfdffb8e41273fbc287293bd2d0a3e59.jpg',
'/d/file/20150905/ffa892119baef28dadfcadc3b7abd50a.jpg',
'/d/file/20170331/5164f1dc347b2cd5d6b57088d2614c4d.jpg',
'/d/file/299823f15c021ede81f844b30b8d543a.jpg',
'/d/file/bf464364888cfecaf95b0a2a9e3407bf.png',
'/d/file/a9843ed50a56e020ed78e40a348bcb5c.jpg',
'/d/file/f9fe702b5e01210af5afd970e8499090.jpg',
'/d/file/d3e3d2526686463b4a1406e039388ca6.jpg',
'/d/file/1e29bc94ee7ebde0ae50bc2b5bd112c9.jpg',
'/d/file/db4d5e02fdea2e910ee35eb6b4979a99.jpg',
'/d/file/afaf7502ffca5a0bb052eddba8195509.jpg',
'/d/file/90d63585d7230a48ddd9516dbf0dd6ab.jpg',
'/d/file/689fa17c65c8d0f049521383d43a4ab2.jpg',
'/d/file/20151217/b5108ea8a7a4920b2d7bf6e27c0b1101.jpg',
'/d/file/e81bf99a232cdeee60dc65418a8c2602.png',
'/d/file/04c92ce23ee91e9cd857955e4a60b46d.jpg',
'/d/file/9a3315019ae4b15e84e18770f91e7f28.jpg',
'/d/file/2d64c4f98c23b4177ee1ae9725354ea4.jpg',
'/d/file/0ec2c52c6954837c2b588bb61e7e2e15.jpg',
'/d/file/20160220/b1e09b5d13c74f444806086a1a8aedbe.jpg',
'/d/file/6854f7987613ceac3a21049efa2f7047.jpg',
'/d/file/873316b54c2bc3bcc3573971fda73286.png',
'/d/file/117e999a87ef237115071d64b692b134.jpg',
'/d/file/24191b9f48d9f0ee58baf69e13a18c78.jpg',
'/d/file/2568b473a4a93b0be1366800d9194799.jpg',
'/d/file/0a83bccd885cba88a11eb93c5f789035.jpg',
'/d/file/a55cc2a57a2af7fad3477d799215ad05.jpg',
'/d/file/cd38314fd9c336afe56ef8d61cb8416d.jpg',
'/d/file/c0a77ff668ed2bb6292ec81a4c4105c7.jpg',
'/d/file/2bb8f36b6f540360d081ea17dad0d1c6.jpg',
'/d/file/20151213/e6f03eb1bb25208f230338ee76963d73.jpg',
'/d/file/c6bfce8e88fbeeb0ec94ed740ea02657.jpg',
'/d/file/9732937ef6b527bc9374bec637f209b4.jpg',
'/d/file/e0f286f6c4a83c5458c99e98d692a383.png']}

研究善未成功，学习仍将深入

scrapy爬取校花网男神图片保存到本地的更多相关文章

  1. python爬虫基础应用----爬取校花网视频

    一.爬虫简单介绍 爬虫是什么? 爬虫是首先使用模拟浏览器访问网站获取数据,然后通过解析过滤获得有价值的信息,最后保存到到自己库中的程序. 爬虫程序包括哪些模块? python中的爬虫程序主要包括,re ...

  2. Go语言实战-爬取校花网图片

    一.目标网站分析 爬取校花网http://www.xiaohuar.com/大学校花所有图片. 经过分析,所有图片分为四个页面,http://www.xiaohuar.com/list-1-0.htm ...

  3. python实战项目 — 爬取 校花网图片

    重点: 1.  指定路径创建文件夹,判断是否存在 2. 保存图片文件 # 获得校花网的地址,图片的链接 import re import requests import time import os ...

  4. Scrapy爬虫框架之爬取校花网图片

    Scrapy Scrapy是一个为了爬取网站数据,提取结构性数据而编写的应用框架. 其可以应用在数据挖掘,信息处理或存储历史数据等一系列的程序中.其最初是为了页面抓取 (更确切来说, 网络抓取 )所设 ...

  5. Python的scrapy之爬取链家网房价信息并保存到本地

    因为有在北京租房的打算,于是上网浏览了一下链家网站的房价,想将他们爬取下来,并保存到本地. 先看链家网的源码..房价信息 都保存在 ul 下的li 里面 ​ 爬虫结构: ​ 其中封装了一个数据库处理模 ...

  6. 一文搞定scrapy爬取众多知名技术博客文章保存到本地数据库，包含：cnblog、csdn、51cto、itpub、jobbole、oschina等

    本文旨在通过爬取一系列博客网站技术文章的实践,介绍一下scrapy这个python语言中强大的整站爬虫框架的使用.各位童鞋可不要用来干坏事哦,这些技术博客平台也是为了让我们大家更方便的交流.学习.提高 ...

  7. 第六篇 - bs4爬取校花网

    环境:python3  pycharm 模块:requests  bs4  urlretrieve  os  time 第一步:获取网页源代码 import requests from bs4 imp ...

  8. Python-爬取校花网视频(单线程和多线程版本)

    一.参考文章 python爬虫爬取校花网视频,单线程爬取 爬虫----爬取校花网视频,包含多线程版本 上述两篇文章都是对校花网视频的爬取,由于时间相隔很久了,校花网上的一些视频已经不存在了,因此上述文 ...

  9. Python Scrapy 爬取煎蛋网妹子图实例（一）

    前面介绍了爬虫框架的一个实例,那个比较简单,这里在介绍一个实例 爬取 煎蛋网 妹子图,遗憾的是 上周煎蛋网还有妹子图了,但是这周妹子图变成了 随手拍, 不过没关系,我们爬图的目的是为了加强实战应用,管 ...

随机推荐

  1. LintCode翻转字符串问题 - python实现

    题目描述:试实现一个函数reverseWords,该函数传入参数是一个字符串,返回值是单词间做逆序调整后的字符串(只做单词顺序的调整即可). 例如:传入参数为"the sky is blue ...

  2. PL/SQL Developer从11.0.6版本开始32/64为之区分

    PL/SQL Developer从11.0.6版本开始32/64为之区分 在PL/SQL Developer11.0.6版本之前,64位Windows操作系统在使用PL/SQL Developer都未 ...

  3. 【机器学习】逻辑回归（Logistic Regression）

    注:最近开始学习<人工智能>选修课,老师提纲挈领的介绍了一番,听完课只了解了个大概,剩下的细节只能自己继续摸索. 从本质上讲:机器学习就是一个模型对外界的刺激(训练样本)做出反应,趋利避害 ...

  4. 数据分析之pandas模块

    一.Series 类似于一位数组的对象,第一个参数为数据,第二个参数为索引(索引可以不指定,就默认用隐式索引) Series(data=np.random.randint(1,50,(10,))) S ...

  5. 深入浅出 JVM GC（4）常用 GC 参数介绍

    # 前言 从前面的3篇文章中,我们分析了5个垃圾收集器,还有一些 GC 的算法,那么,在 GC 调优中,我们肯定会先判断哪里出现的问题,然后再根据出现的问题进行调优,而调优的手段就是 JVM 提供给我 ...

  6. JVM（一）—— 内存管理

    JVM 内存结构 Java 虚拟机的内存空间分为 5 个部分: 程序计数器 Java 虚拟机栈 本地方法栈 堆 方法区 程序计数器(PC) 为什么需要程序计数器 因为Java虚拟机的多线程是通过线程轮 ...

  7. CentOS 7 安装 .Net Core 2.0 详细步骤

    轰轰烈烈的Core 热潮,从部署环境开始.参照了网上不少前辈的教程,也遇到不少的坑,这边做个完整的笔记. 一.构建.Net core 2的应用程web发布,因为是用来测试centos上的core 环境 ...

  8. LeetCode链表相加-Python<二>

    上一篇:LeetCode两数之和-Python<一> 题目:https://leetcode-cn.com/problems/add-two-numbers/description/ 给定 ...

  9. Java基础——Servlet（八）文件上传下载

    一.简单的文件上传常见的组件Smartupload , Apache 的 commons FileUploadSmartupload上传的步骤: 1.初始化上传上下文 2.准备上传 3.保存文件 &l ...

  10. 最新版本elasticsearch本地搭建入门篇

    最新版本elasticsearch本地搭建入门篇 项目介绍 最近工作用到elasticsearch,主要是用于网站搜索,和应用搜索. 工欲善其事,必先利其器. 自己开始关注elasticsearch, ...