爬虫——python——百度地图经纬度查询——经纬度查看地点地名——利用百度API获取地名经纬度——爬取所有的中国地址
import requests
address = '40.8587960,86.866991'
url = 'http://api.map.baidu.com/geocoder?output=json&key=f247cdb592eb43ebac6ccd27f796e2d2&location=' + str(address)
response = requests.get(url)
answer = response.json()
print('得到反解数据', answer)
使用python爬虫
import requests
address = '40.8587960,86.866991'
url = 'http://api.map.baidu.com/geocoder?output=json&key=f247cdb592eb43ebac6ccd27f796e2d2&location=' + str(address)
response = requests.get(url)
answer = response.json()
print('得到反解数据', answer)
lng = answer['result']['location']['lng']
lat = answer['result']['location']['lat']
formatted_address = answer['result']['formatted_address']
business = answer['result']['business']
city = answer['result']['addressComponent']['city']
direction = answer['result']['addressComponent']['city']
distance = answer['result']['addressComponent']['direction']
district = answer['result']['addressComponent']['district']
province = answer['result']['addressComponent']['province']
street = answer['result']['addressComponent']['street']
street_number = answer['result']['addressComponent']['street_number']
cityCode = answer['result']['cityCode']
s = str(lng) + '|' + str(lat)+ '|' + str(formatted_address) + '|' + str(business) + '|'+ str(city) + '|'+ str(direction) + '|'+ str(distance) + '|'+ str(district) + '|'+ str(province) + '|'+ str(street) + '|'+ str(street_number) + '|'+ str(cityCode)
print(s)
解析数据
http://api.map.baidu.com/lbsapi/getpoint/index.html
#鸡头1
#129.000000,54.000000 #锚点(鸡头)
#115.000000,54.0000000
#115.000000,42.0000000 #北京
#129.000000,42.000000 #沈阳
#['54.0000000', '42.000000', '129.000000', '115.000000'],#鸡头1 #鸡头2
#129.000000,42.000000 #沈阳
#135.200000,42.000000
#129.000000,50.000000
#135.200000,50.000000
#['50.000000','42.000000','135.000000','129.000000'],#鸡头2
#鸡头补充
#115.000000,42.0000000 #北京
#105.0000000,42.000000 #锚点(银川上面)
#105.0000000,45.500000 #乌兰巴托
#115.000000,45.500000
#['45.500000', '42.000000', '115.000000', '105.000000'],#鸡头补充 #鸡屁股上补充
#105.0000000,42.000000 #锚点(银川上面)
#91.500000,45.500000 #乌鲁木齐
#105.0000000,45.500000 #乌兰巴托
#91.500000,42.000000 #锚点(鸡屁股)
#['45.500000', '42.000000', '105.0000000', '91.500000'],#鸡屁股上补充
#鸡屁股上
#91.500000,42.000000 #锚点(鸡屁股)
#79.800000,42.000000 #啊拉木图
#79.800000,49.200000
#91.500000,49.200000
#['49.200000', '42.000000', '91.500000', '79.800000'],#鸡屁股上
#鸡屁股下
#79.800000,42.000000 #啊拉木图
#79.800000,30.0000000 #巴雷利
#73.400000,42.000000
#73.400000,30.000000
#['42.000000', '30.000000', '79.800000', '73.400000'],#鸡屁股下
#鸡下补充
#79.800000,30.0000000 #巴雷利
#97.300000,30.000000 #拉萨昌都(锚点)
#97.300000,26.800000
#79.800000,26.800000
#['42.000000', '30.000000', '97.300000', '79.800000'],#鸡下补充
#鸡下
#97.300000,30.000000 #拉萨昌都(锚点)
#105.000000,30.000000 #(成都重庆贵阳)
#105.000000,21.000000 #河内
#97.300000,21.000000 #曼德勒
#['30.000000', '21.000000', '105.000000', '97.300000'],#鸡下 #鸡胸
#105.0000000,42.000000 #锚点(银川上面)
#105.0000000,21.000000 #河内
#129.000000,42.000000 #沈阳
#129.000000,21.000000 #台北(海)
#['42.000000', '21.000000', '129.000000', '105.000000'],#鸡胸 #鸡身
#105.000000,42.000000 #锚点(银川上面)
#79.800000,42.000000 #啊拉木图
#79.800000,30.000000 #巴雷利
#105.000000,30.000000 #(成都重庆贵阳)
#['42.000000', '30.000000', '105.000000', '79.800000'],#鸡身
#鸡脚 (南海)
#105.000000,21.000000 #河内
#129.000000,21.000000 #台北(海)
#105.0000000,3.000000 #新加坡
#129.000000,3.000000 #海(苏拉威西海)
#['21.000000', '3.000000', '129.000000', '105.000000'],
经纬度分析
import requests
from decimal import Decimal
def obtain(address):
url = 'http://api.map.baidu.com/geocoder?output=json&key=f247cdb592eb43ebac6ccd27f796e2d2&location=' + str(address)
response = requests.get(url)
answer = response.json()
print('得到反解数据', answer)
lng = answer['result']['location']['lng']
lat = answer['result']['location']['lat']
formatted_address = answer['result']['formatted_address']
business = answer['result']['business']
city = answer['result']['addressComponent']['city']
direction = answer['result']['addressComponent']['city']
distance = answer['result']['addressComponent']['direction']
district = answer['result']['addressComponent']['district']
province = answer['result']['addressComponent']['province']
street = answer['result']['addressComponent']['street']
street_number = answer['result']['addressComponent']['street_number']
cityCode = answer['result']['cityCode']
s = str('%6f' % lng) + '|' + str('%6f' % lat) + '|' + str(formatted_address) + '|' + str(business) + '|' + str(
city) + '|' + str(direction) + '|' + str(distance) + '|' + str(district) + '|' + str(province) + '|' + str(
street) + '|' + str(street_number) + '|' + str(cityCode)
print(s) def longitude(slon,precision,elon,slat):
while slon >= elon:
address = '%s,%s' % (slon,slat) # 请求用的,经度,纬度,
obtain(address)#传入经度,纬度,获取地址地名
slon -= precision #['42.000000', '30.000000', '105.000000', '79.800000'], # 鸡身
slon = Decimal('42.000000')#经度longitude开始
elon = Decimal('30.000000')#经度结束
slat=Decimal('105.000000')# 纬度latitude开始
elat=Decimal('79.800000')#纬度结束
precision = Decimal('5.0001000')#精度 0.0001000==1000米
while slat>=elat:
slat -= precision
longitude(slon,precision,elon,slat)
根据精度获取多个地名
import requests
from decimal import Decimal
def obtain(address):
url = 'http://api.map.baidu.com/geocoder?output=json&key=f247cdb592eb43ebac6ccd27f796e2d2&location=' + str(address)
response = requests.get(url)
answer = response.json()
print('得到反解数据', answer)
lng = answer['result']['location']['lng']
lat = answer['result']['location']['lat']
formatted_address = answer['result']['formatted_address']
business = answer['result']['business']
city = answer['result']['addressComponent']['city']
direction = answer['result']['addressComponent']['city']
distance = answer['result']['addressComponent']['direction']
district = answer['result']['addressComponent']['district']
province = answer['result']['addressComponent']['province']
street = answer['result']['addressComponent']['street']
street_number = answer['result']['addressComponent']['street_number']
cityCode = answer['result']['cityCode']
s = str('%6f' % lng) + '|' + str('%6f' % lat) + '|' + str(formatted_address) + '|' + str(business) + '|' + str(
city) + '|' + str(direction) + '|' + str(distance) + '|' + str(district) + '|' + str(province) + '|' + str(
street) + '|' + str(street_number) + '|' + str(cityCode)
print(s) def longitude(slon,precision,elon,slat):
while slon >= elon:
address = '%s,%s' % (slon,slat) # 请求用的,经度,纬度,
obtain(address)#传入经度,纬度,获取地址地名
slon -= precision LogLatList=[
['42.000000', '30.000000', '105.000000', '79.800000'], # 鸡身
['42.000000', '21.000000', '129.000000', '105.000000'], # 鸡胸
['50.000000', '42.000000', '135.000000','129.000000'], #鸡头2
['54.000000', '42.000000', '129.000000', '115.000000'], #鸡头1
['45.500000', '42.000000', '115.000000', '105.000000'], #鸡头补充
['49.200000', '42.000000', '91.500000', '79.800000'], # 鸡屁股上
['42.000000', '30.000000', '79.800000', '73.400000'], # 鸡屁股下
['45.500000', '42.000000', '105.000000', '91.500000'], #鸡屁股上补充
['30.000000', '21.000000', '105.000000', '97.300000'], # 鸡下
['42.000000', '30.000000', '97.300000', '79.800000'], #鸡下补充
['21.000000', '3.000000', '129.000000', '105.000000'], # 鸡脚 (南海)
]
precision = Decimal('10.0001000') # 精度 0.0001000==1000米
for single_list in LogLatList:
slons,elons,slats,elats=single_list
#['42.000000', '30.000000', '105.000000', '79.800000'], # 鸡身
slon = Decimal(slons)#经度longitude开始
elon = Decimal(elons)#经度结束
slat=Decimal(slats)# 纬度latitude开始
elat=Decimal(elats)#纬度结束 while slat>=elat:
slat -= precision
longitude(slon,precision,elon,slat)
批量获取数据
import requests
from decimal import Decimal
import threading , time
def obtain(address):
url = 'http://api.map.baidu.com/geocoder?output=json&key=f247cdb592eb43ebac6ccd27f796e2d2&location=' + str(address)
response = requests.get(url)
answer = response.json()
print('得到反解数据', answer)
lng = answer['result']['location']['lng']
lat = answer['result']['location']['lat']
formatted_address = answer['result']['formatted_address']
business = answer['result']['business']
city = answer['result']['addressComponent']['city']
direction = answer['result']['addressComponent']['city']
distance = answer['result']['addressComponent']['direction']
district = answer['result']['addressComponent']['district']
province = answer['result']['addressComponent']['province']
street = answer['result']['addressComponent']['street']
street_number = answer['result']['addressComponent']['street_number']
cityCode = answer['result']['cityCode']
s = str('%6f' % lng) + '|' + str('%6f' % lat) + '|' + str(formatted_address) + '|' + str(business) + '|' + str(
city) + '|' + str(direction) + '|' + str(distance) + '|' + str(district) + '|' + str(province) + '|' + str(
street) + '|' + str(street_number) + '|' + str(cityCode)
print(s) def longitude(slon,precision,elon,slat):
while slon >= elon:
address = '%s,%s' % (slon,slat) # 请求用的,经度,纬度,
obtain(address)#传入经度,纬度,获取地址地名
slon -= precision class Thre(threading.Thread):#继承线程中的类
#Thre(single_list,precision,ctime)
def __init__(self,single_list,precision,ctime):
super(Thre,self).__init__()#重新写父类,解决多继承问题
self.precision=precision#精度
self.single_list=single_list#单列表
self.ctime=ctime#当前时间
def run(self):
print('执行线程开始时间:', self.ctime, '执行的列表:',self.single_list)
slons, elons, slats, elats = self.single_list
# ['42.000000', '30.000000', '105.000000', '79.800000'], # 鸡身
slon = Decimal(slons) # 经度longitude开始
elon = Decimal(elons) # 经度结束
slat = Decimal(slats) # 纬度latitude开始
elat = Decimal(elats) # 纬度结束
while slat >= elat:
slat -= self.precision
longitude(slon,self.precision, elon, slat)
else:
end_time=time.time()-self.ctime
print('执行线程所用时间:',end_time, '执行的列表:',self.single_list) def main():
LogLatList=[
['42.000000', '30.000000', '105.000000', '79.800000'], # 鸡身
['42.000000', '21.000000', '129.000000', '105.000000'], # 鸡胸
['50.000000', '42.000000', '135.000000','129.000000'], #鸡头2
['54.000000', '42.000000', '129.000000', '115.000000'], #鸡头1
['45.500000', '42.000000', '115.000000', '105.000000'], #鸡头补充
['49.200000', '42.000000', '91.500000', '79.800000'], # 鸡屁股上
['42.000000', '30.000000', '79.800000', '73.400000'], # 鸡屁股下
['45.500000', '42.000000', '105.000000', '91.500000'], #鸡屁股上补充
['30.000000', '21.000000', '105.000000', '97.300000'], # 鸡下
['42.000000', '30.000000', '97.300000', '79.800000'], #鸡下补充
['21.000000', '3.000000', '129.000000', '105.000000'], # 鸡脚 (南海)
]
precision = Decimal('20.0001000') # 精度 0.0001000==1000米 thre_list=[] #线程列表
for single_list in LogLatList:
ctime=time.time()#当前时间
temp=Thre(single_list,precision,ctime)#实例化类
thre_list.append(temp)#线程列表
for thre in thre_list:#线程列表
thre.start()#执行单个线程 if __name__ == '__main__':
main()
多线程获取数据
import requests
from decimal import Decimal
import threading , time
def obtain(address,file_name):
url = 'http://api.map.baidu.com/geocoder?output=json&key=f247cdb592eb43ebac6ccd27f796e2d2&location=' + str(address)
response = requests.get(url)
answer = response.json()
print('得到反解数据', answer)
lng = answer['result']['location']['lng']
lat = answer['result']['location']['lat']
formatted_address = answer['result']['formatted_address']
business = answer['result']['business']
city = answer['result']['addressComponent']['city']
direction = answer['result']['addressComponent']['city']
distance = answer['result']['addressComponent']['direction']
district = answer['result']['addressComponent']['district']
province = answer['result']['addressComponent']['province']
street = answer['result']['addressComponent']['street']
street_number = answer['result']['addressComponent']['street_number']
cityCode = answer['result']['cityCode']
s = str('%6f' % lng) + '|' + str('%6f' % lat) + '|' + str(formatted_address) + '|' + str(business) + '|' + str(
city) + '|' + str(direction) + '|' + str(distance) + '|' + str(district) + '|' + str(province) + '|' + str(
street) + '|' + str(street_number) + '|' + str(cityCode)
print(s)
if cityCode==0:
#print('外国')
pass
else:
#print(s)
fname=str(file_name)+'list'
with open(fname, 'a+', encoding=('utf-8')) as f:
f.write(s+'\n') def longitude(slon,precision,elon,slat,file_name):
while slon >= elon:
address = '%s,%s' % (slon,slat) # 请求用的,经度,纬度,
obtain(address,file_name)#传入经度,纬度,获取地址地名
slon -= precision class Thre(threading.Thread):#继承线程中的类
#Thre(single_list,precision,ctime)
#temp = Thre(single_list, precision, ctime, file_name) # 实例化类
def __init__(self,single_list,precision,ctime,file_name):
super(Thre,self).__init__()#重新写父类,解决多继承问题
self.precision=precision#精度
self.single_list=single_list#单列表
self.ctime=ctime#当前时间
self.file_name=file_name#文件名
def run(self):
print('执行线程开始时间:', self.ctime, '执行的列表:',self.single_list)
slons, elons, slats, elats = self.single_list
# ['42.000000', '30.000000', '105.000000', '79.800000'], # 鸡身
slon = Decimal(slons) # 经度longitude开始
elon = Decimal(elons) # 经度结束
slat = Decimal(slats) # 纬度latitude开始
elat = Decimal(elats) # 纬度结束
while slat >= elat:
slat -= self.precision
longitude(slon,self.precision, elon, slat,self.file_name)
else:
end_time=time.time()-self.ctime
print('执行线程所用时间:',end_time, '执行的列表:',self.single_list) def main():
LogLatList=[
['42.000000', '30.000000', '105.000000', '79.800000'], # 鸡身
['42.000000', '21.000000', '129.000000', '105.000000'], # 鸡胸
['50.000000', '42.000000', '135.000000','129.000000'], #鸡头2
['54.000000', '42.000000', '129.000000', '115.000000'], #鸡头1
['45.500000', '42.000000', '115.000000', '105.000000'], #鸡头补充
['49.200000', '42.000000', '91.500000', '79.800000'], # 鸡屁股上
['42.000000', '30.000000', '79.800000', '73.400000'], # 鸡屁股下
['45.500000', '42.000000', '105.000000', '91.500000'], #鸡屁股上补充
['30.000000', '21.000000', '105.000000', '97.300000'], # 鸡下
['42.000000', '30.000000', '97.300000', '79.800000'], #鸡下补充
['21.000000', '3.000000', '129.000000', '105.000000'], # 鸡脚 (南海)
]
# ——测试使用——开始————
precision = Decimal('1.0001000') # 精度 0.0001000==1000米
# ——测试使用——结束———— #——实际获取时替换——精度自己控制————建议是 0.0001000 ————开始————
#precision = Decimal('0.0001000') # 精度 0.0001000==1000米
# ——实际获取时替换——精度自己控制————建议是 0.0001000 ————结束———— file_name=0 thre_list=[] #线程列表
for single_list in LogLatList:
ctime=time.time()#当前时间
temp=Thre(single_list,precision,ctime,file_name)#实例化类
file_name+=1
thre_list.append(temp)#线程列表
for thre in thre_list:#线程列表
thre.start()#执行单个线程 if __name__ == '__main__':
main()
写入到文件
爬虫——python——百度地图经纬度查询——经纬度查看地点地名——利用百度API获取地名经纬度——爬取所有的中国地址的更多相关文章
- Android百度地图开发-第一篇:申请、搭建百度地图
一.前言 这是第一篇关于Android使用百度地图的学习记录,主要记录: 1.在百度地图开发者平台上申请API Key. 2.在自己的应用中加入百度地图的Android版SDK. 3.在自己的应用中显 ...
- 百度地图JavaScript API获取用户当前经纬度和详细地理位置,反之通过详细地理位置获取当前经纬度
前言: 前端时间刚好使用了百度地图的js api定位获取用户当前经纬度并获取当前详细位置和通过当前用户详细地理位置换取用户当前经纬度坐标的功能,为了方便下次找起来方便一些自己在这里记录一下,希望也能够 ...
- python爬虫学习(二):定向爬虫例子-->使用BeautifulSoup爬取"软科中国最好大学排名-生源质量排名2018",并把结果写进txt文件
在正式爬取之前,先做一个试验,看一下爬取的数据对象的类型是如何转换为列表的: 写一个html文档: x.html<html><head><title>This is ...
- Python 从底层结构聊 Beautiful Soup 4(内置豆瓣最新电影排行榜爬取案例)
1. 前言 什么是 Beautiful Soup 4 ? Beautiful Soup 4(简称 BS4,后面的 4 表示最新版本)是一个 Python 第三方库,具有解析 HTML 页面的功能,爬虫 ...
- 针对源代码和检查元素不一致的网页爬虫——利用Selenium、PhantomJS、bs4爬取12306的列车途径站信息
整个程序的核心难点在于上次豆瓣爬虫针对的是静态网页,源代码和检查元素内容相同:而在12306的查找搜索过程中,其网页发生变化(出现了查找到的数据),这个过程是动态的,使得我们在审查元素中能一一对应看到 ...
- 简单又强大的pandas爬虫 利用pandas库的read_html()方法爬取网页表格型数据
文章目录 一.简介 二.原理 三.爬取实战 实例1 实例2 一.简介 一般的爬虫套路无非是发送请求.获取响应.解析网页.提取数据.保存数据等步骤.构造请求主要用到requests库,定位提取数据用的比 ...
- Python 爬取图书图片和地址
#-*- coding:utf-8 -*- import xlwt import urllib import re def getHtml(url): page = urllib.urlopen(ur ...
- Python反爬:利用js逆向和woff文件爬取猫眼电影评分信息
首先:看看运行结果效果如何! 1. 实现思路 小编基本实现思路如下: 利用js逆向模拟请求得到电影评分的页面(就是猫眼电影的评分信息并不是我们上述看到的那个页面上,应该它的实现是在一个页面上插入另外一 ...
- 记录开发基于百度地图API实现在地图上绘制轨迹并拾取轨迹对应经纬度的工具说明
前言: 最近一直在做数据可视化方面的工作,其中平面可视化没什么难度,毕竟已经有很多成熟的可供使用的框架,比如百度的echart.js,highcharts.js等.还有就是3D可视化了,整体来说难度也 ...
随机推荐
- Quartz2作业监听
在本教程中,我们将展示/介绍如何创建一个JobListener,跟踪运行工作状态在作业完成等. P.S 这个例子是Quartz 2.1.5 1. Quartz 作业 作业 - 用于打印一个简单的信息, ...
- Android Telephony分析(四) ---- TelephonyManager详解
前言 TelephonyManager主要提供Telephony相关信息的查询/修改功能,以及Phone状态监听功能,封装的方法主要是提供给APP上层使用.TelephonyManager.java ...
- adostoredproc用法 因为用的少每次还得看一下代码,记下来
{1.关闭2.清除参数(固定的可省略)3.参数赋值4.打开(或执行)如果有感知控件的话 就会显示出结果} ADOStoredProc1.close; //关闭 ADOStoredProc1.param ...
- Codeforces 479【D】div3
题目链接:http://codeforces.com/problemset/problem/977/D 题意:给你一个数字序列,定了一个游戏规则.你可以对当前数字进行两个操作 1./ 3 如果这个数 ...
- Docker的概念及基本用法
Docker是PaaS供应商dotCloud开源的一个基于LXC 的高级容器引擎,源代码托管在 GitHub 上, 基于Go语言开发并遵从Apache 2.0协议开源.Docker提供了一种在安全.可 ...
- Linux sed命令实现替换文本内容
/root/data/code-s3201/publish_codex/deploy/db.properties db.properties中的 1.0.0.6 替换为 1.0.0.7 sed -i ...
- merge存储引擎基本介绍和使用规范说明
[以下两段截自MySql手册]: merge存储引擎,也被认识为mrg_myisam引擎,是一个"相同“的表可以被当作一个表来用的myisam表的集合.“相同”意味着所有表同样的列和索引信息 ...
- Java迷宫代码,广度优先遍历,最短路径
使用一个队列,采用层层扩张的方式,寻找迷宫最优的路径信息,再用一个迷宫节点数组记录行走信息方向常量定义: public interface Constant { // 右方向 int RIGHT = ...
- 导出Excel格式数据
/** * 导出规则统计数据 excel格式 * @param bill * @return */ public String exportExamsignupExamnoExcel ( String ...
- create table常用命令
CREATE TABLE students( stuID INTEGER NOT NULL , stuname ) not null, sex int NOT NULL ); CREATE TABLE ...