bs4——BeautifulSoup模块：解析网页

解析由requests模块请求到的网页

 import requests

 from bs4 import BeautifulSoup

 headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/\

 537.36 (KHTML, like Gecko) Chrome/70.0.3538.67 Safari/537.36',

                'Host':'movie.douban.com'}

 link = 'https://movie.douban.com/top250'

 r = requests.get(link, headers=headers,timeout=2)

 soup = BeautifulSoup(r.text, 'lxml') #以lxml格式解析网页文本

BeautifulSoup模块有两个查找方法：

一个是：find()，返回符合条件的第一条内容

 import requests

 from bs4 import BeautifulSoup

 headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/\

 537.36 (KHTML, like Gecko) Chrome/70.0.3538.67 Safari/537.36',

                'Host':'movie.douban.com'}

 link = 'https://movie.douban.com/top250'

 r = requests.get(link, headers=headers,timeout=2)

 soup = BeautifulSoup(r.text, 'lxml') #以lxml格式解析网页文本

 find_result = soup.find('div', class_='hd')

 print(find_result)

下面是解析后的结果：

C:\python3.5\python.exe C:/Users/MR/Desktop/test.py

<div class="hd">

<a class="" href="https://movie.douban.com/subject/1292052/">

<span class="title">肖申克的救赎</span>

<span class="title"> / The Shawshank Redemption</span>

<span class="other"> / 月黑高飞(港)  /  刺激1995(台)</span>

</a>

<span class="playable">[可播放]</span>

</div>

另一个是：find_all()，以列表格式返回符合条件的所有内容

 import requests

 from bs4 import BeautifulSoup

 headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/\

 537.36 (KHTML, like Gecko) Chrome/70.0.3538.67 Safari/537.36',

                'Host':'movie.douban.com'}

 link = 'https://movie.douban.com/top250'

 r = requests.get(link, headers=headers,timeout=2)

 soup = BeautifulSoup(r.text, 'lxml') #以lxml格式解析网页文本

 find_result = soup.find_all('div', class_='hd')

 print(find_result)

下面是解析后结果：

　　注意返回的是列表

C:\python3.5\python.exe C:/Users/MR/Desktop/test.py

[<div class="hd">

<a class="" href="https://movie.douban.com/subject/1292052/">

<span class="title">肖申克的救赎</span>

<span class="title"> / The Shawshank Redemption</span>

<span class="other"> / 月黑高飞(港)  /  刺激1995(台)</span>

</a>

<span class="playable">[可播放]</span>

</div>, <div class="hd">

<a class="" href="https://movie.douban.com/subject/1291546/">

<span class="title">霸王别姬</span>

<span class="other"> / 再见，我的妾  /  Farewell My Concubine</span>

</a>

<span class="playable">[可播放]</span>

</div>, <div class="hd">

<a class="" href="https://movie.douban.com/subject/1295644/">

<span class="title">这个杀手不太冷</span>

<span class="title"> / Léon</span>

<span class="other"> / 杀手莱昂  /  终极追杀令(台)</span>

</a>

<span class="playable">[可播放]</span>

</div>, <div class="hd">

<a class="" href="https://movie.douban.com/subject/1292720/">

<span class="title">阿甘正传</span>

<span class="title"> / Forrest Gump</span>

<span class="other"> / 福雷斯特·冈普</span>

</a>

<span class="playable">[可播放]</span>

</div>, <div class="hd">

<a class="" href="https://movie.douban.com/subject/1292063/">

<span class="title">美丽人生</span>

<span class="title"> / La vita è bella</span>

<span class="other"> / 一个快乐的传说(港)  /  Life Is Beautiful</span>

</a>

<span class="playable">[可播放]</span>

</div>, <div class="hd">

<a class="" href="https://movie.douban.com/subject/1292722/">

<span class="title">泰坦尼克号</span>

<span class="title"> / Titanic</span>

<span class="other"> / 铁达尼号(港 / 台)</span>

</a>

<span class="playable">[可播放]</span>

</div>, <div class="hd">

<a class="" href="https://movie.douban.com/subject/1291561/">

<span class="title">千与千寻</span>

<span class="title"> / 千と千尋の神隠し</span>

<span class="other"> / 神隐少女(台)  /  Spirited Away</span>

</a>

</div>, <div class="hd">

<a class="" href="https://movie.douban.com/subject/1295124/">

<span class="title">辛德勒的名单</span>

<span class="title"> / Schindler's List</span>

<span class="other"> / 舒特拉的名单(港)  /  辛德勒名单</span>

</a>

<span class="playable">[可播放]</span>

</div>, <div class="hd">

<a class="" href="https://movie.douban.com/subject/3541415/">

<span class="title">盗梦空间</span>

<span class="title"> / Inception</span>

<span class="other"> / 潜行凶间(港)  /  全面启动(台)</span>

</a>

<span class="playable">[可播放]</span>

</div>, <div class="hd">

<a class="" href="https://movie.douban.com/subject/2131459/">

<span class="title">机器人总动员</span>

<span class="title"> / WALL·E</span>

<span class="other"> / 瓦力(台)  /  太空奇兵·威E(港)</span>

</a>

<span class="playable">[可播放]</span>

</div>, <div class="hd">

<a class="" href="https://movie.douban.com/subject/3011091/">

<span class="title">忠犬八公的故事</span>

<span class="title"> / Hachi: A Dog's Tale</span>

<span class="other"> / 忠犬小八(台)  /  秋田犬八千(港)</span>

</a>

<span class="playable">[可播放]</span>

</div>, <div class="hd">

<a class="" href="https://movie.douban.com/subject/3793023/">

<span class="title">三傻大闹宝莱坞</span>

<span class="title"> / 3 Idiots</span>

<span class="other"> / 三个傻瓜(台)  /  作死不离3兄弟(港)</span>

</a>

</div>, <div class="hd">

<a class="" href="https://movie.douban.com/subject/1292001/">

<span class="title">海上钢琴师</span>

<span class="title"> / La leggenda del pianista sull'oceano</span>

<span class="other"> / 声光伴我飞(港)  /  一九零零的传奇</span>

</a>

<span class="playable">[可播放]</span>

</div>, <div class="hd">

<a class="" href="https://movie.douban.com/subject/1291549/">

<span class="title">放牛班的春天</span>

<span class="title"> / Les choristes</span>

<span class="other"> / 歌声伴我心(港)  /  唱诗班男孩</span>

</a>

<span class="playable">[可播放]</span>

</div>, <div class="hd">

<a class="" href="https://movie.douban.com/subject/1292213/">

<span class="title">大话西游之大圣娶亲</span>

<span class="title"> / 西遊記大結局之仙履奇緣</span>

<span class="other"> / 西游记完结篇仙履奇缘  /  齐天大圣西游记</span>

</a>

<span class="playable">[可播放]</span>

</div>, <div class="hd">

<a class="" href="https://movie.douban.com/subject/1292064/">

<span class="title">楚门的世界</span>

<span class="title"> / The Truman Show</span>

<span class="other"> / 真人Show(港)  /  真人戏</span>

</a>

<span class="playable">[可播放]</span>

</div>, <div class="hd">

<a class="" href="https://movie.douban.com/subject/1291841/">

<span class="title">教父</span>

<span class="title"> / The Godfather</span>

<span class="other"> / Mario Puzo's The Godfather</span>

</a>

<span class="playable">[可播放]</span>

</div>, <div class="hd">

<a class="" href="https://movie.douban.com/subject/1889243/">

<span class="title">星际穿越</span>

<span class="title"> / Interstellar</span>

<span class="other"> / 星际启示录(港)  /  星际效应(台)</span>

</a>

<span class="playable">[可播放]</span>

</div>, <div class="hd">

<a class="" href="https://movie.douban.com/subject/1291560/">

<span class="title">龙猫</span>

<span class="title"> / となりのトトロ</span>

<span class="other"> / 邻居托托罗  /  邻家的豆豆龙</span>

</a>

</div>, <div class="hd">

<a class="" href="https://movie.douban.com/subject/5912992/">

<span class="title">熔炉</span>

<span class="title"> / 도가니</span>

<span class="other"> / 无声呐喊(港)  /  漩涡</span>

</a>

</div>, <div class="hd">

<a class="" href="https://movie.douban.com/subject/1307914/">

<span class="title">无间道</span>

<span class="title"> / 無間道</span>

<span class="other"> / Infernal Affairs  /  Mou gaan dou</span>

</a>

<span class="playable">[可播放]</span>

</div>, <div class="hd">

<a class="" href="https://movie.douban.com/subject/1849031/">

<span class="title">当幸福来敲门</span>

<span class="title"> / The Pursuit of Happyness</span>

<span class="other"> / 寻找快乐的故事(港)  /  追求快乐</span>

</a>

<span class="playable">[可播放]</span>

</div>, <div class="hd">

<a class="" href="https://movie.douban.com/subject/6786002/">

<span class="title">触不可及</span>

<span class="title"> / Intouchables</span>

<span class="other"> / 闪亮人生(港)  /  逆转人生(台)</span>

</a>

</div>, <div class="hd">

<a class="" href="https://movie.douban.com/subject/3319755/">

<span class="title">怦然心动</span>

<span class="title"> / Flipped</span>

<span class="other"> / 萌动青春  /  青春萌动</span>

</a>

<span class="playable">[可播放]</span>

</div>, <div class="hd">

<a class="" href="https://movie.douban.com/subject/1300267/">

<span class="title">乱世佳人</span>

<span class="title"> / Gone with the Wind</span>

<span class="other"> / 飘</span>

</a>

<span class="playable">[可播放]</span>

</div>]

bs4——BeautifulSoup模块：解析网页的更多相关文章

Python学习－使用BeautifulSoup来解析网页一：基础入门
写技术博客主要就是总结和交流的,如果文章用错,请指正啊! 以前一直在使用SGMLParser,这个太费时间和精力了,现在为了毕业设计,改用BeautifulSoup来实现HTML页面的解析工作的. 一 ...
使用BeautifulSoup高效解析网页,再也不用担心睡不着觉了
BeautifulSoup是一个可以从 HTML 或 XML 文件中提取数据的 Python 库那需要怎么使用呢? 首先我们要安装一下这个库 1.pip install beautifulsoup4 ...
requests和BeautifulSoup模块的使用
用python写爬虫时,有两个很好用第三方模块requests库和beautifulsoup库,简单学习了下模块用法: 1,requests模块 Python标准库中提供了:urllib.urllib ...
python网络爬虫之解析网页的BeautifulSoup(爬取电影图片)[三]
目录前言一.BeautifulSoup的基本语法二.爬取网页图片扩展学习后记前言本章同样是解析一个网页的结构信息在上章内容中(python网络爬虫之解析网页的正则表达式(爬取4k动漫图 ...
【爬虫入门手记03】爬虫解析利器beautifulSoup模块的基本应用
[爬虫入门手记03]爬虫解析利器beautifulSoup模块的基本应用 1.引言网络爬虫最终的目的就是过滤选取网络信息,因此最重要的就是解析器了,其性能的优劣直接决定这网络爬虫的速度和效率.Bea ...
爬虫模块介绍--Beautifulsoup （解析库模块，正则）
Beautiful Soup 是一个可以从HTML或XML文件中提取数据的Python库.它能够通过你喜欢的转换器实现惯用的文档导航,查找,修改文档的方式.Beautiful Soup会帮你节省数小时 ...
【网络爬虫入门03】爬虫解析利器beautifulSoup模块的基本应用
[网络爬虫入门03]爬虫解析利器beautifulSoup模块的基本应用 1.引言网络爬虫最终的目的就是过滤选取网络信息,因此最重要的就是解析器了,其性能的优劣直接决定这网络爬虫的速度和效率.B ...
03 解析库之Beautifulsoup模块
Beautifulsoup模块一介绍 Beautiful Soup 是一个可以从HTML或XML文件中提取数据的Python库.它能够通过你喜欢的转换器实现惯用的文档导航,查找,修改文档的方式 ...
python BeautifulSoup模块的简要介绍
常用介绍: pip install beautifulsoup4 # 安装模块 from bs4 import BeautifulSoup # 导入模块 soup = BeautifulSoup(ht ...

随机推荐

WPF实现系统禁音的方法
方法1: [DllImport("user32.dll", CharSet = CharSet.Auto, SetLastError = true)] static extern ...
【必须知道】Enum_Flags
[Flags] enum AnyThings{ A=1, B=2, C=4, D=8 } 枚举赋值必须是2^n才可以,目的是实现他们的二进制表示中的 1 ,不要重叠,如 1=0001 2=0010 ...
log4j-slf4j 典型用例
一.maven 配置 <dependency> <groupId>org.slf4j</groupId> <artifactId>slf4j-log4j ...
Codility---Dominator
Task description A zero-indexed array A consisting of N integers is given. The dominator of array A ...
发布一个Django项目
一.部署环境准备,准备python3和虚拟环境解释器,virtualenvwrapper 1.修改python3的环境变量 PATH=/opt/python36/bin:/usr/local/sbin ...
never下的easysql
什么是EasySql 在我们早期写的代码中,想实现组装灵活的sql语句与参数,我们可以去翻阅早期自己写的代码 var @sb = new StringBuilder(); sb.Append(&quo ...
Go - Slice 切片
概述切片是一种动态数组,比数组操作灵活,长度不是固定的,可以进行追加和删除. len() 和 cap() 返回结果可相同和不同. 声明切片 //demo_7.go package main impo ...
ASP.NET Core on K8S学习初探（2）K8S基本概念快速一览
在上一篇<单节点环境搭建>中,通过Docker for Windows在Windows开发机中搭建了一个单节点的K8S环境,接下来就是动人心弦的部署ASP.NET Core API到K8S ...
Jenkins Email Extension插件模板
Jenkins Email Extension插件模板 <!DOCTYPE html> <html> <head> <meta charset="U ...
【朝花夕拾】Android自定义View篇之（九）多点触控（下）实践出真知
前言在上一篇文章中,已经总结了MotionEvent以及多点触控相关的基础理论知识和常用的函数.本篇将通过实现单指拖动图片,多指拖动图片的实际案例来进行练习并实现一些效果,来理解前面的理论知识.要理 ...

bs4——BeautifulSoup模块：解析网页

bs4——BeautifulSoup模块：解析网页的更多相关文章

随机推荐

热门专题