多线程下载英文Google地图
1. pom.xml文件
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion> <groupId>com.smartmap</groupId>
<artifactId>sano3</artifactId>
<version>1.0-SNAPSHOT</version> <properties>
<junit.version>4.12</junit.version>
<log4j.version>1.2.17</log4j.version>
<slf4j.version>1.7.21</slf4j.version>
<commmons.collections.version>3.2.1</commmons.collections.version>
<commons.fileupload.version>1.2.1</commons.fileupload.version>
<commons.io.version>2.4</commons.io.version> <!-- Encoding -->
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<maven.compiler.encoding>UTF-8</maven.compiler.encoding>
</properties> <dependencies>
<!-- https://mvnrepository.com/artifact/org.xerial/sqlite-jdbc -->
<dependency>
<groupId>org.xerial</groupId>
<artifactId>sqlite-jdbc</artifactId>
<version>3.15.1</version>
</dependency> <!-- https://mvnrepository.com/artifact/org.apache.httpcomponents/httpclient -->
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.5.2</version>
</dependency>
<dependency>
<groupId>dom4j</groupId>
<artifactId>dom4j</artifactId>
<version>1.6.1</version>
<scope>test</scope>
</dependency>
<dependency>
<!-- General data-binding functionality for Jackson: works on core streaming API -->
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>2.8.4</version>
</dependency>
<dependency>
<!-- Json -->
<groupId>org.json</groupId>
<artifactId>json</artifactId>
<version>20160810</version>
</dependency>
<dependency>
<!-- Core Protocol Buffers library. Protocol Buffers are a way of encoding structured data in an efficient yet extensible format. -->
<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-java</artifactId>
<version>3.1.0</version>
</dependency> <dependency>
<!-- A plugin to allow execution of system and Java programs -->
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.5.0</version>
</dependency>
<dependency>
<!-- Runs Ant scripts embedded in the POM -->
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-antrun-plugin</artifactId>
<version>1.8</version>
</dependency>
<dependency>
<!-- The Resources Plugin handles the copying of project resources to the output directory. There are two different kinds of resources: main resources and test resources. -->
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-resources-plugin</artifactId>
<version>3.0.1</version>
</dependency>
<dependency>
<!-- Builds a Java Archive (JAR) file from the compiled project classes and resources. -->
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>3.0.2</version>
</dependency>
<dependency>
<!-- The Compiler Plugin is used to compile the sources of your project. -->
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.5.1</version>
</dependency>
<dependency>
<!-- Repackages the project classes together with their dependencies into a single uber-jar, optionally renaming classes or removing unused classes. -->
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.4.3</version>
</dependency>
<dependency>
<!-- The Maven Source Plugin creates a JAR archive of the source files of the current project. -->
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-source-plugin</artifactId>
<version>3.0.1</version>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>${log4j.version}</version>
<exclusions>
<exclusion>
<groupId>com.sun.jdmk</groupId>
<artifactId>jmxtools</artifactId>
</exclusion>
<exclusion>
<groupId>com.sun.jmx</groupId>
<artifactId>jmxri</artifactId>
</exclusion>
</exclusions>
<scope>compile</scope>
</dependency>
<dependency>
<!-- The slf4j API -->
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<!--<version>1.7.21</version>-->
<version>1.5.5</version>
</dependency>
<dependency>
<!-- SLF4J LOG4J-12 Binding -->
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<!--<version>1.7.21</version>-->
<version>1.5.5</version>
</dependency>
<dependency>
<!-- Maven Surefire MOJO in maven-surefire-plugin -->
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19.1</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>${junit.version}</version>
<scope>test</scope>
</dependency>
</dependencies> <build>
<sourceDirectory>src/main/java</sourceDirectory>
<testSourceDirectory>src/test/java</testSourceDirectory>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.5.0</version>
<executions>
<execution>
<goals>
<goal>exec</goal>
</goals>
</execution>
</executions>
<configuration>
<executable>java</executable>
<arguments>
<argument>-classpath</argument>
<classpath>
</classpath>
<argument>com.smartmap.exam.salon3.App</argument>
</arguments>
</configuration>
</plugin>
</plugins>
</build> </project>
2. App.java
package com.smartmap.exam.salon3; import com.smartmap.exam.salon3.tilecache.DownTileMutipleThreadApp;
import com.smartmap.exam.salon3.tilecache.DownTileSingleThreadApp; /**
* Created by zyx on 2016/11/5.
*/
public class App {
public int add(int a, int b) {
return a + b;
}
public int subtract(int a, int b) {
return a - b;
} public App() {
} public static void main(String args[]) {
System.out.println("Hello World!"); int startLevel = 1;
int endLevel = 7;
double xmin = -180;
double ymin = -85.05112878;
double xmax = 180;
double ymax = 85.05112878; //DownTileSingleThreadApp downTile = new DownTileSingleThreadApp();
//downTile.downTile(startLevel, endLevel, xmin, ymin, xmax, ymax); //
/*
DownTileSingleThreadApp downTile = new DownTileSingleThreadApp();
String[] parameterStringArray = {
"16,22,-70.7863,-33.5632,-70.5662,-33.3624"
};
//
for(int i=0; i<parameterStringArray.length; i++) {
String parameterString = parameterStringArray[i];
String[] parameters = parameterString.split(",");
System.out.println(parameters);
startLevel = Integer.valueOf(parameters[0]);
endLevel = Integer.valueOf(parameters[1]);
xmin = Double.valueOf(parameters[2]);
ymin = Double.valueOf(parameters[3]);
xmax = Double.valueOf(parameters[4]);
ymax = Double.valueOf(parameters[5]);
downTile.downTile(startLevel, endLevel, xmin, ymin, xmax, ymax);
}
*/
/*
"10,15,-74.2393899661748,-35.2486992939983,-67.7829962222134,-31.5372209203788",
"1,9,-180,-85.05112878,180,85.05112878",
"7,11,-78.7627850312735,-38.6775577375241,-63.0565026784138,-28.8170274076513",
*/
String[] parameterStringArray = {
"22,23,-70.7863,-33.5632,-70.5662,-33.3624",
"9,13,-80.7848,-39.5913,-61.9966,-26.9706" };
//
DownTileMutipleThreadApp downTile = new DownTileMutipleThreadApp();
for(int i=0; i<parameterStringArray.length; i++) {
String parameterString = parameterStringArray[i];
String[] parameters = parameterString.split(",");
System.out.println(parameters);
startLevel = Integer.valueOf(parameters[0]);
endLevel = Integer.valueOf(parameters[1]);
xmin = Double.valueOf(parameters[2]);
ymin = Double.valueOf(parameters[3]);
xmax = Double.valueOf(parameters[4]);
ymax = Double.valueOf(parameters[5]);
for(int level=startLevel; level<endLevel; level++) {
System.out.println("------------------------------" + level + "-----------------------");
for(int k=0; k<2; k++) { downTile.downTileRange(level, xmin, ymin, xmax, ymax);
System.out.println("------------------------------" + level + "-----------------------");
}
}
}
}
}
3. DownTileMutipleThreadApp.java
package com.smartmap.exam.salon3.tilecache; import org.slf4j.Logger;
import org.slf4j.LoggerFactory; /**
* Created by zyx on 2016/11/28.
*/
public class DownTileMutipleThreadApp {
private static final Logger logger = LoggerFactory.getLogger(DownTileMutipleThreadApp.class);
private final static double xmin = -180.0;
private final static double ymin = -85.05112878;
private final static double xmax = 180;
private final static double ymax = 85.05112878;
//
private final static double minXPro = -20037508.3427892;
private final static double minYPro = -20037508.3427892;
private final static double maxXPro = 20037508.3427892;
private final static double maxYPro = 20037508.3427892;
//
private final static double earthRadius = 6378137.0;
//
private String baseURL = "http://www.google.cn/maps/vt?lyrs=m%40189&gl=cn";
private String baseFilePath = "E:\\google\\en\\map";
//
private int threadCountRow = 10;
private int threadCountColumn = 10;
/**
*
* @param xmin
* @param ymin
* @param xmax
* @param ymax
* @return
*/
private double[] computeProjectCoordinate(double xmin, double ymin, double xmax, double ymax) {
double xminProject = earthRadius * (xmin - 0) / 180 * Math.PI;
double yminProject = earthRadius * Math.log(Math.tan(Math.PI / 4 + ymin / 2 * Math.PI / 180));
double xmaxProject = earthRadius * (xmax - 0) / 180 * Math.PI;
double ymaxProject = earthRadius * Math.log(Math.tan(Math.PI / 4 + ymax / 2 * Math.PI / 180));
//print(xminProject, yminProject, xmaxProject, ymaxProject)
double[] projectCoordinateRange = new double[]{xminProject, yminProject, xmaxProject, ymaxProject};
return projectCoordinateRange;
} /**
*
* @param level
* @param xminProject
* @param yminProject
* @param xmaxProject
* @param ymaxProject
* @return
*/
private int[] computeTileRange(int level, double xminProject, double yminProject, double xmaxProject, double ymaxProject) {
double mapXYSpan = (maxXPro - minXPro);
double mapImageXYSpan = mapXYSpan / Math.pow(2.0, level); int rowStart = (int)((maxYPro - ymaxProject) / mapImageXYSpan);
int rowEnd = (int)((maxYPro - yminProject) / mapImageXYSpan); int columnStart = (int)((xminProject - minXPro) / mapImageXYSpan)+1;
int columnEnd = (int)((xmaxProject - minXPro) / mapImageXYSpan)+1;
int[] tileRange = new int[]{columnStart, rowStart, columnEnd, rowEnd};
//
System.out.println(mapImageXYSpan + " " + rowStart + " " + rowEnd + " " + columnStart + " " + columnEnd);
//
return tileRange;
} public void downTileRange(int level, double xmin, double ymin, double xmax, double ymax){
//
try {
double[] projectCoordinateRange = computeProjectCoordinate(xmin, ymin, xmax, ymax);
int[] tileRange = computeTileRange(level, projectCoordinateRange[0], projectCoordinateRange[1], projectCoordinateRange[2], projectCoordinateRange[3]);
//
int columnStart = tileRange[0];
int rowStart = tileRange[1];
int columnEnd = tileRange[2];
int rowEnd = tileRange[3];
//
int columnCountAll = columnEnd - columnStart;
int rowCountAll = rowEnd - rowStart;
//
int columnCountPerThread = columnCountAll / this.threadCountColumn;
columnCountPerThread += ((columnCountAll % this.threadCountColumn == 0)?0:1);
//
int rowCountPerThread = rowCountAll / this.threadCountRow;
rowCountPerThread += ((rowCountAll % this.threadCountRow == 0)?0:1);
System.out.println(columnCountPerThread + " <----> " + rowCountPerThread);
//
ThreadGroup threadGroup = new ThreadGroup("DownTileMutipleThreadAppThreadGroup");
//
DownTileThread downTileThread = null;
//
for(int i = 0; i<this.threadCountRow; i++) {
int rowStartThread = rowStart + i * rowCountPerThread;
int rowEndThread = rowStart + (i + 1) * rowCountPerThread;
if(rowStartThread > rowEnd)
{
break;
}
if(rowEndThread > rowEnd)
{
rowEndThread = rowEnd;
}
for (int j = 0; j < this.threadCountColumn; j++) {
int columnStartThread = columnStart + j * columnCountPerThread;
int columnEndThread = columnStart + (j + 1) * columnCountPerThread;
if(columnStartThread > columnEnd)
{
break;
}
if(columnEndThread > columnEnd)
{
columnEndThread = columnEnd;
}
// String baseFilePath, int level, int columnStart, int rowStart, int columnEnd, int rowEnd
String threadName = "Thread" + i * j;
downTileThread = new DownTileThread(threadGroup, threadName, this.baseFilePath, level, columnStartThread, rowStartThread, columnEndThread, rowEndThread);
downTileThread.start();
//Thread.currentThread().sleep(3000);
}
}
Thread.currentThread().sleep(3000);
downTileThread.join();
int activeCount = threadGroup.activeCount();
while(activeCount > 0)
{
Thread.currentThread().sleep(5000);
activeCount = threadGroup.activeCount();
System.out.println("activeCount: " + activeCount);
}
} catch (InterruptedException e) {
e.printStackTrace();
}
} //8,10,-74.2393899661748,-35.2486992939983,-67.7829962222134,-31.5372209203788
//7,9,-78.7627850312735,-38.6775577375241,-63.0565026784138,-28.8170274076513
//6,8,-100.725925780599,-50.7810948395102,-29.9193824693404,-8.25138729930426
// 10,14,-71.4579,-34.1396,-69.9808,-32.7919
public static void main(String args[]) {
int startLevel = 1;
int endLevel = 7;
double xmin = -180;
double ymin = -85.05112878;
double xmax = 180;
double ymax = 85.05112878; //1,9,-180,-85.05112878,180,85.05112878
DownTileMutipleThreadApp downTile = new DownTileMutipleThreadApp();
/*
Scanner scanner = new Scanner(System.in);
System.out.println("input: startLevel endLevel xmin ymin xmax ymax"); String parameterString = scanner.next(); "6,9,-129.217225609756,-60.7335151345493,-4.98429878048784,12.9539853357581",
"7,9,-78.7627850312735,-38.6775577375241,-63.0565026784138,-28.8170274076513",
"8,10,-74.2393899661748,-35.2486992939983,-67.7829962222134,-31.5372209203788",
"10,16,-70.7863,-33.5632,-70.5662,-33.3624",
"10,13,-71.4579,-34.1396,-69.9808,-32.7919",
"13,16,-71.0329,-33.8309,-70.4218,-33.1669",
*/
//"6,8,-100.725925780599,-50.7810948395102,-29.9193824693404,-8.25138729930426",
String[] parameterStringArray = {
"19,21,-70.7863,-33.5632,-70.5662,-33.3624"
};
for(int i=0; i<parameterStringArray.length; i++) {
String parameterString = parameterStringArray[i];
String[] parameters = parameterString.split(",");
System.out.println(parameters);
startLevel = Integer.valueOf(parameters[0]);
endLevel = Integer.valueOf(parameters[1]);
xmin = Double.valueOf(parameters[2]);
ymin = Double.valueOf(parameters[3]);
xmax = Double.valueOf(parameters[4]);
ymax = Double.valueOf(parameters[5]);
for(int level=startLevel; level<endLevel; level++) {
System.out.println("------------------------------" + level + "-----------------------");
downTile.downTileRange(level, xmin, ymin, xmax, ymax);
}
}
}
}
4. DownTileSingleThreadApp.java
package com.smartmap.exam.salon3.tilecache; import org.apache.http.HttpEntity;
import org.apache.http.HttpStatus;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.config.Registry;
import org.apache.http.config.RegistryBuilder;
import org.apache.http.conn.socket.ConnectionSocketFactory;
import org.apache.http.conn.socket.LayeredConnectionSocketFactory;
import org.apache.http.conn.socket.PlainConnectionSocketFactory;
import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
import org.apache.http.conn.ssl.SSLContexts;
import org.apache.http.conn.ssl.TrustStrategy;
import org.apache.http.impl.client.*; import javax.net.ssl.SSLContext; import java.io.BufferedOutputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.security.KeyManagementException;
import java.security.KeyStore;
import java.security.KeyStoreException;
import java.security.NoSuchAlgorithmException;
import java.security.cert.CertificateException;
import java.security.cert.X509Certificate;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.methods.HttpGet; import org.apache.http.impl.conn.PoolingHttpClientConnectionManager;
import org.apache.http.util.EntityUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory; import java.io.File;
import java.util.Iterator;
import java.util.Map; /**
* Created by zyx on 2016/11/21.
*/
public class DownTileSingleThreadApp {
private static final Logger logger = LoggerFactory.getLogger(DownTileSingleThreadApp.class);
private final static double xmin = -180.0;
private final static double ymin = -85.05112878;
private final static double xmax = 180;
private final static double ymax = 85.05112878;
//
private final static double minXPro = -20037508.3427892;
private final static double minYPro = -20037508.3427892;
private final static double maxXPro = 20037508.3427892;
private final static double maxYPro = 20037508.3427892;
//
private final static double earthRadius = 6378137.0;
//
private String baseURL = "http://www.google.cn/maps/vt?lyrs=m%40189&gl=cn";
private String baseFilePath = "E:\\google\\en\\map";
// /**
*
* @param xmin
* @param ymin
* @param xmax
* @param ymax
* @return
*/
private double[] computeProjectCoordinate(double xmin, double ymin, double xmax, double ymax) {
double xminProject = earthRadius * (xmin - 0) / 180 * Math.PI;
double yminProject = earthRadius * Math.log(Math.tan(Math.PI / 4 + ymin / 2 * Math.PI / 180));
double xmaxProject = earthRadius * (xmax - 0) / 180 * Math.PI;
double ymaxProject = earthRadius * Math.log(Math.tan(Math.PI / 4 + ymax / 2 * Math.PI / 180));
//print(xminProject, yminProject, xmaxProject, ymaxProject)
double[] projectCoordinateRange = new double[]{xminProject, yminProject, xmaxProject, ymaxProject};
return projectCoordinateRange;
} /**
*
* @param level
* @param xminProject
* @param yminProject
* @param xmaxProject
* @param ymaxProject
* @return
*/
private int[] computeTileRange(int level, double xminProject, double yminProject, double xmaxProject, double ymaxProject) {
double mapXYSpan = (maxXPro - minXPro);
double mapImageXYSpan = mapXYSpan / Math.pow(2.0, level); int rowStart = (int)((maxYPro - ymaxProject) / mapImageXYSpan);
int rowEnd = (int)((maxYPro - yminProject) / mapImageXYSpan); int columnStart = (int)((xminProject - minXPro) / mapImageXYSpan);
int columnEnd = (int)((xmaxProject - minXPro) / mapImageXYSpan);
int[] tileRange = new int[]{columnStart, rowStart, columnEnd, rowEnd};
//
System.out.println(mapImageXYSpan + " " + rowStart + " " + rowEnd + " " + columnStart + " " + columnEnd);
//
return tileRange;
} /**
*
* @param level
* @param columnStart
* @param rowStart
* @param columnEnd
* @param rowEnd
*/
private void downTileRange(int level, int columnStart, int rowStart, int columnEnd, int rowEnd) {
//
/*for(int row=rowStart; row<rowEnd; row++)
{
for(int column=columnStart+1; column<columnEnd+1; column++)
{*/
for(int row=rowEnd; row> rowStart-1; row--)
{
for(int column=columnStart; column<columnEnd+1; column++)
{
String url = baseURL + "&z=" + level + "&x=" + column + "&y=" + row;
url = "https://www.google.com:443/maps/vt/pb=!1m4!1m3!1i" + level + "!2i" + column + "!3i" + row + "!2m3!1e0!2sm!3i368045507!3m7!2sen!5e1105!12m4!1e68!2m2!1sset!2sRoadmap!4e0!5m1!1e0";
String fullFilePath = baseFilePath + "\\" + level + "\\" + column + "\\" + row + ".png"; File file = new File(fullFilePath);
System.out.println("check: " + level + " " + column + " " + row);
if(!file.exists())
{
System.out.println("--> " + level + " " + column + " " + row);
System.out.println(url);
File fileParent = file.getParentFile();
if(!fileParent.exists())
{
fileParent.mkdirs();
}
downTileSingle(url, fullFilePath);
}
}
}
} private static CloseableHttpClient getHttpClient() {
RegistryBuilder<ConnectionSocketFactory> registryBuilder = RegistryBuilder.<ConnectionSocketFactory>create();
ConnectionSocketFactory connectionSocketFactory = new PlainConnectionSocketFactory();
registryBuilder.register("http", connectionSocketFactory);
//指定信任密钥存储对象和连接套接字工厂
try {
KeyStore keyStore = KeyStore.getInstance(KeyStore.getDefaultType());
//信任任何链接
TrustStrategy trustStrategy = new TrustStrategy() {
//@Override
public boolean isTrusted(X509Certificate[] x509Certificates, String s) throws CertificateException {
return true;
}
};
SSLContext sslContext = SSLContexts.custom().useTLS().loadTrustMaterial(keyStore, trustStrategy).build();
LayeredConnectionSocketFactory layeredConnectionSocketFactory = new SSLConnectionSocketFactory(sslContext, SSLConnectionSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER);
registryBuilder.register("https", layeredConnectionSocketFactory);
} catch (KeyStoreException e) {
throw new RuntimeException(e);
} catch (KeyManagementException e) {
throw new RuntimeException(e);
} catch (NoSuchAlgorithmException e) {
throw new RuntimeException(e);
}
Registry<ConnectionSocketFactory> registry = registryBuilder.build();
//设置连接管理器
PoolingHttpClientConnectionManager poolingHttpClientConnectionManager = new PoolingHttpClientConnectionManager(registry);
// connManager.setDefaultConnectionConfig(connConfig);
// connManager.setDefaultSocketConfig(socketConfig);
//构建客户端
CloseableHttpClient closeableHttpClient = HttpClientBuilder.create().setConnectionManager(poolingHttpClientConnectionManager).build();
return closeableHttpClient;
} /**
* get
*
* @param url 请求的url
* @param queries 请求的参数，在浏览器？后面的数据，没有可以传null
* @return
* @throws IOException
*/
public static byte[] get(String url, Map<String, String> queries) throws IOException {
byte[] image = null;
//支持https
CloseableHttpClient httpClient = getHttpClient();
StringBuilder stringBuilder = new StringBuilder(url);
if (queries != null && queries.keySet().size() > 0) {
boolean firstFlag = true;
Iterator iterator = queries.entrySet().iterator();
while (iterator.hasNext()) {
Map.Entry entry = (Map.Entry<String, String>) iterator.next();
if (firstFlag) {
stringBuilder.append("?" + (String) entry.getKey() + "=" + (String) entry.getValue());
firstFlag = false;
} else {
stringBuilder.append("&" + (String) entry.getKey() + "=" + (String) entry.getValue());
}
}
} try {
HttpGet httpGet = new HttpGet(stringBuilder.toString());
RequestConfig requestConfig = RequestConfig.custom()
.setSocketTimeout(30000)
.setConnectTimeout(30000).build();//设置请求和传输超时时间
httpGet.setConfig(requestConfig);
//System.out.println("Executing request " + httpGet.getRequestLine());
//请求数据
CloseableHttpResponse response = httpClient.execute(httpGet);
//System.out.println(response.getStatusLine());
int status = response.getStatusLine().getStatusCode();
if (status == HttpStatus.SC_OK) {
HttpEntity entity = response.getEntity();
//String responseBody = EntityUtils.toString(entity);
//EntityUtils.consume(entity);
image = EntityUtils.toByteArray(entity);
} else {
System.out.println("http return status error:" + status);
throw new ClientProtocolException("Unexpected response status: " + status);
}
} catch (Exception ex) {
ex.printStackTrace();
} finally {
httpClient.close();
}
return image;
} public static byte[] sendGetRequest(String url) throws IOException {
byte[] image = null;
//支持https
CloseableHttpClient httpClient = getHttpClient();
HttpGet httpGet = new HttpGet(url);
RequestConfig requestConfig = RequestConfig.custom()
.setSocketTimeout(30000)
.setConnectTimeout(30000).build();//设置请求和传输超时时间
httpGet.setConfig(requestConfig);
try {
//请求数据
CloseableHttpResponse response = httpClient.execute(httpGet);
int status = response.getStatusLine().getStatusCode();
if (HttpStatus.SC_OK == status) {
HttpEntity httpEntity = response.getEntity();
image = EntityUtils.toByteArray(httpEntity);
} else {
System.out.println("http return status error:" + status);
}
} catch (Exception ex) {
ex.printStackTrace();
} finally {
httpClient.close();
}
return image;
} private void saveByteArrayToImage(byte[] image, String filePath)
{
try {
if(image != null)
{
File file = new File(filePath);
//System.out.println(filePath);
//file.createNewFile();
FileOutputStream fileOutputStream = new FileOutputStream(file);
BufferedOutputStream bufferedOutputStream = new BufferedOutputStream(fileOutputStream);
bufferedOutputStream.write(image);
//bufferedOutputStream.flush();
bufferedOutputStream.close();
fileOutputStream.close();
}
} catch (IOException e) {
e.printStackTrace();
}
} /**
*
* @param url
* @param filePath
*/
private void downTileSingle(String url, String filePath) {
byte[] image = null;
try {
image = DownTileSingleThreadApp.sendGetRequest(url);
saveByteArrayToImage(image, filePath);
} catch (IOException e) {
e.printStackTrace();
}
} /**
*
*/
public void downTile(int startLevel, int endLevel, double xmin, double ymin, double xmax, double ymax)
{
for(int level=startLevel; level < endLevel; level++)
{
System.out.println("------------------------------------------");
System.out.println(level);
//
double[] projectCoordinateRange = computeProjectCoordinate(xmin, ymin, xmax, ymax);
int[] tileRange = computeTileRange(level, projectCoordinateRange[0], projectCoordinateRange[1], projectCoordinateRange[2], projectCoordinateRange[3]); downTileRange(level, tileRange[0], tileRange[1], tileRange[2], tileRange[3]);
System.out.println("------------------------------------------");
}
}
//8,10,-74.2393899661748,-35.2486992939983,-67.7829962222134,-31.5372209203788
//7,9,-78.7627850312735,-38.6775577375241,-63.0565026784138,-28.8170274076513
//6,8,-100.725925780599,-50.7810948395102,-29.9193824693404,-8.25138729930426
// 10,14,-71.4579,-34.1396,-69.9808,-32.7919
public static void main(String args[]) {
int startLevel = 1;
int endLevel = 7;
double xmin = -180;
double ymin = -85.05112878;
double xmax = 180;
double ymax = 85.05112878; //
DownTileSingleThreadApp downTile = new DownTileSingleThreadApp();
/*
Scanner scanner = new Scanner(System.in);
System.out.println("input: startLevel endLevel xmin ymin xmax ymax"); String parameterString = scanner.next();
*/
//"6,8,-100.725925780599,-50.7810948395102,-29.9193824693404,-8.25138729930426",
String[] parameterStringArray = {
"6,9,-129.217225609756,-60.7335151345493,-4.98429878048784,12.9539853357581",
"7,9,-78.7627850312735,-38.6775577375241,-63.0565026784138,-28.8170274076513",
"8,10,-74.2393899661748,-35.2486992939983,-67.7829962222134,-31.5372209203788",
"10,16,-70.7863,-33.5632,-70.5662,-33.3624",
"10,13,-71.4579,-34.1396,-69.9808,-32.7919",
"13,16,-71.0329,-33.8309,-70.4218,-33.1669",
"16,21,-70.7863,-33.5632,-70.5662,-33.3624"
};
for(int i=0; i<parameterStringArray.length; i++) {
String parameterString = parameterStringArray[i];
String[] parameters = parameterString.split(",");
System.out.println(parameters);
startLevel = Integer.valueOf(parameters[0]);
endLevel = Integer.valueOf(parameters[1]);
xmin = Double.valueOf(parameters[2]);
ymin = Double.valueOf(parameters[3]);
xmax = Double.valueOf(parameters[4]);
ymax = Double.valueOf(parameters[5]);
downTile.downTile(startLevel, endLevel, xmin, ymin, xmax, ymax);
}
} }
5. DownTileThread.java
package com.smartmap.exam.salon3.tilecache; import org.apache.http.HttpEntity;
import org.apache.http.HttpStatus;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.config.Registry;
import org.apache.http.config.RegistryBuilder;
import org.apache.http.conn.socket.ConnectionSocketFactory;
import org.apache.http.conn.socket.LayeredConnectionSocketFactory;
import org.apache.http.conn.socket.PlainConnectionSocketFactory;
import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
import org.apache.http.conn.ssl.SSLContextBuilder;
import org.apache.http.conn.ssl.SSLContexts;
import org.apache.http.conn.ssl.TrustStrategy;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClientBuilder;
import org.apache.http.impl.conn.PoolingHttpClientConnectionManager;
import org.apache.http.util.EntityUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory; import javax.net.ssl.SSLContext;
import java.io.BufferedOutputStream;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.security.KeyManagementException;
import java.security.KeyStore;
import java.security.KeyStoreException;
import java.security.NoSuchAlgorithmException;
import java.security.cert.CertificateException;
import java.security.cert.X509Certificate; /**
* Created by zyx on 2016/11/28.
*/
public class DownTileThread extends Thread { private static final Logger logger = LoggerFactory.getLogger(DownTileSingleThreadApp.class); //
//private String baseURL = "http://www.google.cn/maps/vt?lyrs=m%40189&gl=cn";
//
private String baseFilePath = "E:\\google\\en\\map";
private int level;
private int columnStart;
private int rowStart;
private int columnEnd;
private int rowEnd; /**
*
* @param baseFilePath
* @param level
* @param columnStart
* @param rowStart
* @param columnEnd
* @param rowEnd
*/
public DownTileThread(ThreadGroup threadGroup, String name, String baseFilePath, int level, int columnStart, int rowStart, int columnEnd, int rowEnd)
{
super(threadGroup, name);
this.baseFilePath = baseFilePath;
this.level = level;
this.columnStart = columnStart;
this.rowStart = rowStart;
this.columnEnd = columnEnd;
this.rowEnd = rowEnd;
} @Override
public void run() {
System.out.println("Thread: " + this.getName() + " " + rowStart + " " + rowEnd + " " + columnStart + " " + columnEnd);
downTileRange();
} private void downTileRange() {
//
try {
CloseableHttpClient httpClient = getHttpClient();
for(int row=rowStart; row<rowEnd; row++)
{
for(int column=columnStart; column<columnEnd; column++)
{
String url = "https://www.google.com:443/maps/vt/pb=!1m4!1m3!1i" + level + "!2i" + column + "!3i" + row + "!2m3!1e0!2sm!3i368045507!3m7!2sen!5e1105!12m4!1e68!2m2!1sset!2sRoadmap!4e0!5m1!1e0";
String fullFilePath = baseFilePath + "\\" + level + "\\" + column + "\\" + row + ".png";
File file = new File(fullFilePath);
if(!file.exists())
{
System.out.println(this.getName()+" :--> " + level + " " + column + " " + row);
//System.out.println(url);
File fileParent = file.getParentFile();
if(!fileParent.exists())
{
fileParent.mkdirs();
}
downTileSingle(httpClient, url, fullFilePath);
}
}
}
if(httpClient != null) {
httpClient.close();
}
} catch (IOException e) {
e.printStackTrace();
}
} /**
*
* @param url
* @param filePath
*/
public void downTileSingle(CloseableHttpClient httpClient, String url, String filePath) {
byte[] image = null;
try {
image = sendGetRequest(httpClient, url);
saveByteArrayToImage(image, filePath);
} catch (IOException e) {
e.printStackTrace();
}
} public byte[] sendGetRequest(CloseableHttpClient httpClient, String url) throws IOException {
byte[] image = null;
RequestConfig.Builder builder = RequestConfig.custom();
builder.setSocketTimeout(30000);
builder.setConnectTimeout(30000);
RequestConfig requestConfig = builder.build();
//支持https
HttpGet httpGet = new HttpGet(url);
httpGet.setConfig(requestConfig);
CloseableHttpResponse response = null;
try {
//请求数据
response = httpClient.execute(httpGet);
int status = response.getStatusLine().getStatusCode();
if (HttpStatus.SC_OK == status) {
HttpEntity httpEntity = response.getEntity();
image = EntityUtils.toByteArray(httpEntity);
} else {
System.out.println("http return status error:" + status);
}
} catch (Exception ex) {
ex.printStackTrace();
} finally {
if(response != null)
{
response.close();
}
}
return image;
} /**
*
* @return
*/
private CloseableHttpClient getHttpClient() {
RegistryBuilder<ConnectionSocketFactory> registryBuilder = RegistryBuilder.<ConnectionSocketFactory>create();
ConnectionSocketFactory connectionSocketFactory = new PlainConnectionSocketFactory();
registryBuilder.register("http", connectionSocketFactory);
//指定信任密钥存储对象和连接套接字工厂
try {
KeyStore keyStore = KeyStore.getInstance(KeyStore.getDefaultType());
//信任任何链接
TrustStrategy trustStrategy = new TrustStrategy() {
//@Override
public boolean isTrusted(X509Certificate[] x509Certificates, String s) throws CertificateException {
return true;
}
};
SSLContextBuilder sslContextBuilder = SSLContexts.custom();
sslContextBuilder.useTLS();
sslContextBuilder.loadTrustMaterial(keyStore, trustStrategy);
SSLContext sslContext = sslContextBuilder.build();
LayeredConnectionSocketFactory layeredConnectionSocketFactory = new SSLConnectionSocketFactory(sslContext, SSLConnectionSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER);
registryBuilder.register("https", layeredConnectionSocketFactory);
} catch (KeyStoreException e) {
throw new RuntimeException(e);
} catch (KeyManagementException e) {
throw new RuntimeException(e);
} catch (NoSuchAlgorithmException e) {
throw new RuntimeException(e);
}
Registry<ConnectionSocketFactory> registry = registryBuilder.build();
//设置连接管理器
PoolingHttpClientConnectionManager poolingHttpClientConnectionManager = new PoolingHttpClientConnectionManager(registry);
// connManager.setDefaultConnectionConfig(connConfig);
// connManager.setDefaultSocketConfig(socketConfig);
//构建客户端
HttpClientBuilder httpClientBuilder = HttpClientBuilder.create();
httpClientBuilder.setConnectionManager(poolingHttpClientConnectionManager);
CloseableHttpClient closeableHttpClient = httpClientBuilder.build();
return closeableHttpClient;
} private void saveByteArrayToImage(byte[] image, String filePath)
{
try {
if(image != null)
{
File file = new File(filePath);
//System.out.println(filePath);
//file.createNewFile();
FileOutputStream fileOutputStream = new FileOutputStream(file);
BufferedOutputStream bufferedOutputStream = new BufferedOutputStream(fileOutputStream);
bufferedOutputStream.write(image);
//bufferedOutputStream.flush();
bufferedOutputStream.close();
fileOutputStream.close();
}
} catch (IOException e) {
e.printStackTrace();
}
}
}
多线程下载英文Google地图的更多相关文章
- 使用QT开发GoogleMap瓦片显示和下载工具（2）——Google地图瓦片投影和获取网址解析
这篇主要说的是Google地图采用的投影方式,瓦片划分方式以及如何从给定的经纬度获取瓦片的数据的网址.所有资料均来自于网络,并亲自试验可行. Google地图投影 首先是地图投影问题,因为地球本身是一 ...
- Flex加载google地图、百度地图以及天地图作底图
一 Flex加载Google地图作底图 (1)帮助类GoogleLayer.as /* * 根据输入的地图类型加载Google地图(by chenyuming) */ package Layers ...
- Python3 多线程下载代码
根据http://www.oschina.net/code/snippet_70229_2407修改而来的增强版.貌似原版源自Axel这个多线程下载工具. ''' Created on 2014-10 ...
- Android中多线程下载列表的封装实现(含进度反馈)
来源:http://blog.csdn.net/u011638883/article/details/17347015 实现了一下Android中的文件多线程下载模块,支持自定义线程数.断点续传.下载 ...
- libcurl的封装，支持同步异步请求，支持多线程下载，支持https
最近在做一个项目,需要用到http get post等 需求分析需要做到同步和异步,异步请求的返回以可选的回调通知的方式进行. 本人以Linux为例,一步一步的来实现. 配置并且编译libcurl我以 ...
- 在VC/MFC中嵌入Google地图——图文并茂
近期须要实验室须要将在无人机地面站中嵌入地图,在网上找了非常多资料,最终有些眉目了, 首先.做这个须要用到的知识有.MFC控件.MFC类库.JavaScript脚本语言.Google API.Goog ...
- 使用GDAL将下载的Google卫星图像转为带坐标的tif
网上有很多下载Google地图的卫片的软件,一般下载下来的图像都是jpg格式的,另外附带一个坐标信息的描述文件.这样的数据不能直接拿来在遥感或者GIS软件中使用,因为图像里面没有投影和坐标信息,所以就 ...
- 网页中嵌入google地图
一丶前言 大致需求:美国地图中标记分布的仓库图钉(鼠标经过显示地址详情),通过输入寄收件地邮编来在地图上定位位置添加图钉,将寄件地,选择的仓库,收件地图钉折线相连接,表示大致路线. 一丶google开 ...
- Android集成Google地图详细步骤记录
先贴下Google官方的地图demo地址:https://github.com/googlemaps/android-samples 那么接下来第一步,申请Google的API key. 使用谷歌账号 ...
随机推荐
- phpqrcode 分别生成普通、带logo、带背景带logo二维码
前提:下载好 phpqrcode 类库 1.生成普通二维码 //引入类库 include('phpqrcode/phpqrcode.php'); //二维码里面的链接地址 $url="htt ...
- 【JXOI2018】排序问题 贪心
我们令$sum_i$表示数字i在加完数字的数列中出现的次数,那么答案显然为$\dfrac{(n+m)!}{\sum_{i=0}^{\infty}sum_i!}$ 不难发现,当每次添加的数为$[l,r] ...
- NOIP2017滚粗记【上】
Day0: NOIP前停课训练的最后一天,上午打了一场三题都见过的比赛,一窝人AK. 下午一群人在机房缓慢氧化,到了晚上因为比赛在我们学校打,所以所有的机房都断网了(百思不得其解为什么两个竞赛室也被断 ...
- VMware workstation 11 的下载
不多说,直接上干货! VMWare Workstation 11的下载详细: 谷歌FQ,进入. 具体怎么达到可以FQ,见 FQ软件lantern-installer-beta.exe(推荐) 成功! ...
- Gen代码解读
源代码为: package com.test19; public class Test04 { void spin(){ int i; for(i=0;i<100;i++){ ; } i = 0 ...
- HTTP状态码列表
经常用的http状态码 1xx消息——请求已被服务器接收,继续处理2xx成功——请求已成功被服务器接收.理解.并接受3xx重定向——需要后续操作才能完成这一请求4xx请求错误——请求含有词法错误或者无 ...
- 打包命令tar
tar是linux下最常用的打包命令,使用tar打出来的包我们常称为tar包,tar包文件的命令通常都是以.tar结尾的,也有.gz或.bz2结尾的. 1 常用参数 -c,--create # 新建打 ...
- java字符串应用之表达式解析器
一.表达式的组成 1.数字 2.运算符:+ - / * ^ % = 3.圆括号 4.变量二.运算符优先级 由高到低分别为:+-(正负号).^.*/%.+-.= 优先 ...
- 前端性能优化---缓存篇SDK
1.把前端最常用的资源css.js存在本地1.1 前端缓存技术SessionStorage 优点:临时存储神器,关闭页面标签自动回收,不可以跨页面交互. 取值的时候有两种方法,一种是用session ...
- 阿里云centos6.5实践编译安装LNMP架构web环境
LNMP 代表的就是:Linux系统下Nginx+MySQL+PHP这种网站服务器架构. 本次测试需求: **实践centos6.5编译安装 LNMP生产环境 架构 web生产环境 使用 ngx_pa ...