Android 多线程下载,断点续传,线程池
你可以在这里看到这个demo的源码:
https://github.com/onlynight/MultiThreadDownloader
效果图
这张效果图是同时开启三个下载任务,限制下载线程数量的效果图。
多线程下载原理
多线程下载的原理就是将下载任务分割成一个个小片段再将每个小片段分配给各个线程进行下载。
例如一个文件大小为100M,我们决定使用4个线程下载,那么每个线程下载的大小即为25M,每个线程的起始以及结束位置依次如下:
0: 0-25M
1: 25-50M
2: 50-75M
3: 75-END
下载请求使用的是Http GET方法,GET请求允许我们设置参数请求的数据的范围:
HttpURLConnection conn = (HttpURLConnection)
new URL("download url").openConnection();
// 设置http为get方法
conn.setRequestMethod("GET");
// 设置http请求的范围
conn.setRequestProperty("Range", "bytes=" + startPos + "-" + endPos);
多线程访问文件需要创建RandomAccessFile:
RandomAccessFile threadFile = new RandomAccessFile(
fileDownloader.getFileSavePath() + File.separator +
fileDownloader.getFilename(), "rwd");
threadFile.seek(startPos);
断点续传原理
端点续传要求我们将之前下载过的文件片段保存下来,并且记录上次最后的下载位置。下次继续下载的时候从上次记录的位置开始下载即可,无需再从头下载。
线程池作用
线程池的原理分析和使用请查看者几篇文章:
http://blog.csdn.net/u010687392/article/details/49850803
http://blog.csdn.net/qq_17250009/article/details/50906508
试想如果我们有100个下载任务,我们让每个任务分成3个线程下载,那么每个任务都需要4个线程,如果100个任务同时开启下载那么就意味着需要同时启动400个线程执行下载任务,这样势必会影响app性能。
像上面这样大量的创建线程的操作势必会影响操作系统的性能,等这些任务执行完成后销毁线程同样也会消耗很多的系统资源,所以Java中提出了线程池的概念。
下面我们来分析线程池是如何节约系统资源的。
从时间角度:
- 创线程的时间我们假定它为: t1
- 线程执行的时间我们假定为: t2
- 销毁线程的时间我们假定为: t3
就用上例我们计算其耗时为:
T = 100(t1+t2+t3)
我们使用固定上限线程数量的线程池耗时为:
T1 = 5(t1+t3)+100t2
显然固定线程上限线程池所需的时间短了很多,固定数量线程池节约了线程创建和销毁的时间,使用线程复用方法避免了线程的频繁创建和销毁,不仅节约了时间同时节约了系统资源。
关键代码
下载线程:
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.io.RandomAccessFile;
import java.net.HttpURLConnection;
import java.net.URL; /**
* Created by lion on 2017/2/10.
*/ public class DownloadRunnable implements Runnable {
private int threadId = -;
private FileDownloader fileDownloader;
private int downloadedSize = ; private int startPos = -;
private int endPos = -;
private int downloadLength = ; private boolean isFinish;
private boolean isStart; public DownloadRunnable(FileDownloader fileDownloader, int threadId, int blockSize,
int downloadedSize) {
this.fileDownloader = fileDownloader;
this.threadId = threadId;
int fileSize = fileDownloader.getFileSize();
this.startPos = blockSize * threadId + downloadedSize;
this.endPos = blockSize * (threadId + ) < fileSize ?
blockSize * (threadId + ) : fileSize;
this.downloadedSize = downloadedSize;
} @Override
public void run() {
if (startPos >= endPos) {
isFinish = true;
} else {
try {
isStart = true;
isFinish = false;
HttpURLConnection conn = (HttpURLConnection)
new URL(fileDownloader.getDownloadUrl()).openConnection();
conn.setConnectTimeout(FileDownloader.getConnectionTimeOut());
conn.setRequestMethod("GET"); //set accept file meta-data type
conn.setRequestProperty("Accept", "image/gif, image/jpeg, image/pjpeg," +
" image/pjpeg, application/x-shockwave-flash, application/xaml+xml, " +
"application/vnd.ms-xpsdocument, application/x-ms-xbap, " +
"application/x-ms-application, application/vnd.ms-excel, " +
"application/vnd.ms-powerpoint, application/msword, */*"); conn.setRequestProperty("Accept-Language", "zh-CN");
conn.setRequestProperty("Referer", fileDownloader.getDownloadUrl());
conn.setRequestProperty("Charset", "UTF-8");
conn.setRequestProperty("User-Agent", "Mozilla/4.0 (compatible; MSIE 8.0; " +
"Windows NT 5.2; Trident/4.0; .NET CLR 1.1.4322; .NET CLR 2.0.50727; " +
".NET CLR 3.0.04506.30; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)");
conn.setRequestProperty("Connection", "Keep-Alive"); conn.setRequestProperty("Range", "bytes=" + startPos + "-" + endPos);
conn.connect(); RandomAccessFile threadFile = new RandomAccessFile(
fileDownloader.getFileSavePath() + File.separator +
fileDownloader.getFilename(), "rwd");
threadFile.seek(startPos);
InputStream inputStream = conn.getInputStream();
byte[] buffer = new byte[];
int offset;
downloadLength = downloadedSize;
while ((offset = inputStream.read(buffer, , )) != -) {
threadFile.write(buffer, , offset);
downloadLength += offset;
fileDownloader.appendDownloadSize(offset);
}
threadFile.close();
inputStream.close(); isFinish = true;
isStart = false;
} catch (IOException e) {
e.printStackTrace();
downloadLength = -;
}
}
} public int getDownloadLength() {
return downloadLength;
} public int getThreadId() {
return threadId;
} public boolean isFinish() {
return isFinish;
} public boolean isStart() {
return isStart;
}
}
下载器:
import android.content.Context;
import android.util.SparseIntArray; import java.io.IOException;
import java.net.HttpURLConnection;
import java.net.URL;
import java.util.Date;
import java.util.UUID;
import java.util.regex.Matcher;
import java.util.regex.Pattern; /**
* Created by lion on 2017/2/7.
*/ public class FileDownloader { public static final String TAG = "FileDownloader"; /**
* http connection timeout
*/
private static int CONNECTION_TIME_OUT = * ; private DownloadProgressManager downloadProgressManager; private DownloadRunnable[] downloadThreads; private String tagName = ""; private String downloadUrl;
private String fileSavePath;
private String filename;
private int threadNum = ;
private int fileSize = ;
private int currentDownloadSize = ; private SparseIntArray currentDownloads; public DownloadRunnable[] getDownloadThreads() {
return downloadThreads;
} public DownloadProgressManager getDownloadProgressManager() {
return downloadProgressManager;
} public int getFileSize() {
return fileSize;
} public String getFileSavePath() {
return fileSavePath;
} public String getFilename() {
return filename;
} public String getDownloadUrl() {
return downloadUrl;
} public int getCurrentDownloadSize() {
return currentDownloadSize;
} public int getThreadNum() {
return threadNum;
} synchronized int appendDownloadSize(int size) {
currentDownloadSize += size;
return currentDownloadSize;
} public FileDownloader(Context context) {
this.currentDownloads = new SparseIntArray();
this.downloadProgressManager = new DownloadProgressManager(context);
} private void requestFileInfo(String downloadUrl) throws RuntimeException {
try {
HttpURLConnection connection = (HttpURLConnection)
new URL(downloadUrl).openConnection();
connection.setConnectTimeout(CONNECTION_TIME_OUT);
connection.setRequestMethod("GET"); //set accept file meta-data type
connection.setRequestProperty("Accept", "image/gif, image/jpeg, image/pjpeg," +
" image/pjpeg, application/x-shockwave-flash, application/xaml+xml, " +
"application/vnd.ms-xpsdocument, application/x-ms-xbap, " +
"application/x-ms-application, application/vnd.ms-excel, " +
"application/vnd.ms-powerpoint, application/msword, */*"); connection.setRequestProperty("Accept-Language", "zh-CN");
connection.setRequestProperty("Referer", downloadUrl);
connection.setRequestProperty("Charset", "UTF-8");
connection.setRequestProperty("User-Agent", "Mozilla/4.0 (compatible; MSIE 8.0; " +
"Windows NT 5.2; Trident/4.0; .NET CLR 1.1.4322; .NET CLR 2.0.50727; " +
".NET CLR 3.0.04506.30; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)");
// connection.setRequestProperty("Connection", "Keep-Alive"); connection.connect(); if (connection.getResponseCode() == ) {
fileSize = connection.getContentLength();
if (fileSize <= ) {
throw new RuntimeException(TAG + " Unknown file size");
} filename = getFilename(connection);
} else {
throw new RuntimeException(TAG + " Server Response Code is "
+ connection.getResponseCode());
}
} catch (IOException e) {
e.printStackTrace();
}
} private String getFilename(HttpURLConnection connection) {
String filename = downloadUrl != null ?
downloadUrl.substring(downloadUrl.lastIndexOf("/") + ) : null;
if (filename == null || "".equals(filename.trim())) {//如果获取不到文件名称
for (int i = ; ; i++) {
String mine = connection.getHeaderField(i);
if (mine == null) break;
if ("content-disposition".equals(connection.getHeaderFieldKey(i).toLowerCase())) {
Matcher m = Pattern.compile(".*filename=(.*)").
matcher(mine.toLowerCase());
if (m.find()) return m.group();
}
}
filename = UUID.randomUUID() + ".tmp";//默认取一个文件名
}
return filename;
} public void prepare(String downloadUrl, String fileSavePath, int threadNum) {
this.downloadUrl = downloadUrl;
this.fileSavePath = fileSavePath;
requestFileInfo(downloadUrl);
SparseIntArray progresses = downloadProgressManager.getProgress(downloadUrl); if (threadNum <= ) {
threadNum = this.threadNum;
} else {
this.threadNum = threadNum;
} if (progresses != null && progresses.size() > ) {
threadNum = progresses.size();
for (int i = ; i < progresses.size(); i++) {
currentDownloadSize += progresses.get(i);
}
} int block = fileSize % threadNum == ?
fileSize / threadNum : fileSize / threadNum + ; downloadThreads = new DownloadRunnable[threadNum]; for (int i = ; i < threadNum; i++) {
downloadThreads[i] = new DownloadRunnable(this, i, block,
progresses != null && progresses.size() == threadNum ?
progresses.valueAt(progresses.keyAt(i)) == - ? :
progresses.valueAt(progresses.keyAt(i)) : );
}
} public void start(OnDownloadListener listener) {
boolean isFinish = false;
int lastDownloadSize = ;
int speed = ;
Date current = new Date();
while (!isFinish) {
if (listener != null) {
int percent = (int) (currentDownloadSize / (float) fileSize * );
long time = new Date().getTime() - current.getTime();
speed = (int) ((currentDownloadSize - lastDownloadSize) / 1024f / time * 1000f);
listener.onUpdate(fileSize, currentDownloadSize, speed, percent);
if (percent == ) {
downloadProgressManager.finishDownload(downloadUrl);
break;
}
}
current = new Date();
lastDownloadSize = currentDownloadSize;
updateProgress();
isFinish = checkFinish();
try {
Thread.sleep();
} catch (InterruptedException e) {
e.printStackTrace();
}
} // System.out.println(tagName + " DOWNLOAD FINISH");
if (listener != null) {
listener.onUpdate(fileSize, fileSize, , );
}
} private boolean checkFinish() {
if (downloadThreads != null && downloadThreads.length > ) {
for (DownloadRunnable downloadThread : downloadThreads) {
if (!downloadThread.isFinish()) {
System.out.println("checkFinish false");
return false;
}
} return true;
}
System.out.println("checkFinish true");
return false;
} public boolean isFinish() {
return checkFinish();
} void updateProgress() {
for (DownloadRunnable downloadThread : downloadThreads) {
updateProgress(downloadThread.getThreadId(), downloadThread.getDownloadLength());
}
} synchronized void updateProgress(int threadId, int downloaded) {
currentDownloads.put(threadId, downloaded);
downloadProgressManager.saveProgress(downloadUrl, currentDownloads);
// SparseIntArray progress = downloadProgressManager.getProgress(downloadUrl);
// for (int i = 0; i < progress.size(); i++) {
// System.out.println("prepare progress = " + progress.valueAt(progress.keyAt(i)));
// }
} public boolean isStart() {
for (DownloadRunnable runnable : downloadThreads) {
if (runnable.isStart()) {
return true;
}
} return false;
} static int getConnectionTimeOut() {
return CONNECTION_TIME_OUT;
} static void setConnectionTimeOut(int timeOut) {
CONNECTION_TIME_OUT = timeOut;
} public interface OnDownloadListener {
void onUpdate(int totalSize, int currentSize, int speed, int percent);
} public void setTagName(String tagName) {
this.tagName = tagName;
}
}
下载管理器:
import android.content.Context; import java.util.ArrayList;
import java.util.concurrent.Executor;
import java.util.concurrent.Executors; /**
* Created by lion on 2017/2/8.
*/ public class DownloadManager { private static int PARALLEL_DOWNLOAD_SIZE = ;
private static DownloadManager instance; private Context context;
private Executor downloadExecutor;
private ArrayList<FileDownloader> fileDownloaders; public static DownloadManager getInstance(Context context) {
if (instance == null) {
instance = new DownloadManager(context);
}
return instance;
} public DownloadManager(Context context) {
this.context = context;
downloadExecutor = Executors.newFixedThreadPool(PARALLEL_DOWNLOAD_SIZE);
// downloadExecutor = Executors.newCachedThreadPool();
fileDownloaders = new ArrayList<>();
} public void download(String name, final String downloadUrl, final String fileSavePath, final int threadNum,
final FileDownloader.OnDownloadListener listener) {
for (FileDownloader downloader : fileDownloaders) {
if (downloader.isFinish()) {
downloader.setTagName(name);
startDownload(downloader, downloadUrl, fileSavePath, threadNum, listener);
return;
}
} FileDownloader currentDownloader = new FileDownloader(context);
currentDownloader.setTagName(name);
fileDownloaders.add(currentDownloader);
startDownload(currentDownloader, downloadUrl, fileSavePath, threadNum, listener);
} public void download(final String downloadUrl, final String fileSavePath, final int threadNum,
final FileDownloader.OnDownloadListener listener) {
for (FileDownloader downloader : fileDownloaders) {
if (downloader.isFinish()) {
startDownload(downloader, downloadUrl, fileSavePath, threadNum, listener);
return;
}
} FileDownloader currentDownloader = new FileDownloader(context);
fileDownloaders.add(currentDownloader);
startDownload(currentDownloader, downloadUrl, fileSavePath, threadNum, listener);
} private synchronized void startDownload(final FileDownloader currentDownloader,
final String downloadUrl, final String fileSavePath,
final int threadNum,
final FileDownloader.OnDownloadListener listener) {
downloadExecutor.execute(new Runnable() {
@Override
public void run() {
currentDownloader.prepare(downloadUrl, fileSavePath,
threadNum);
if (currentDownloader.getDownloadThreads() != null) {
for (DownloadRunnable runnable :
currentDownloader.getDownloadThreads()) {
downloadExecutor.execute(runnable);
}
}
currentDownloader.start(listener);
}
});
} public static void setConnectionTimeOut(int timeOut) {
FileDownloader.setConnectionTimeOut(timeOut);
} public static void setParallelDownloadSize(int size) {
PARALLEL_DOWNLOAD_SIZE = size;
}
}
Android 多线程下载,断点续传,线程池的更多相关文章
- android 多线程下载 断点续传
来源:网易云课堂Android极客班第八次作业练习 练习内容: 多线程 asyncTask handler 多线程下载的原理 首先获取到目标文件的大小,然后在磁盘上申请一块空间用于保存目标文件,接着把 ...
- 【多线程】Android多线程学习笔记——线程池
Java线程池采用了享元设计模式,在系统中维持一定数量的线程,用于处理异步或并发需求,在平时处理异步或并发任务时被广泛使用.这里基于JDK1.8和Android28来整理一些关于线程池的知识点. 一. ...
- android程序---->android多线程下载(一)
多线程下载是加快下载速度的一种方式,通过开启多个线程去执行一个任务,可以使任务的执行速度变快.多线程的任务下载时常都会使用得到断点续传下载,就是我们在一次下载未结束时退出下载,第二次下载时会接着第一次 ...
- android程序---->android多线程下载(二)
上篇我们讲到了android中下载的断点续传问题,今天我们开始学习下载的多线程问题.本次的多线程源码下载:androdi中多线程下载的实现代码.有关断点续传的问题,请参见博客:android程序--- ...
- 无废话Android之smartimageview使用、android多线程下载、显式意图激活另外一个activity,检查网络是否可用定位到网络的位置、隐式意图激活另外一个activity、隐式意图的配置,自定义隐式意图、在不同activity之间数据传递(5)
1.smartimageview使用 <LinearLayout xmlns:android="http://schemas.android.com/apk/res/android&q ...
- Andoid 更好的Android多线程下载框架
概述 为什么是更好的Android多线程下载框架呢,原因你懂的,广告法嘛! 本篇我们我们就来聊聊多线程下载框架,先聊聊我们框架的特点: 多线程 多任务 断点续传 支持大文件 可以自定义下载数据库 高度 ...
- 更好的Android多线程下载框架
/** * 作者:Pich * 原文链接:http://me.woblog.cn/ * QQ群:129961195 * Github:https://github.com/lifengsofts */ ...
- Java多线程系列--“JUC线程池”06之 Callable和Future
概要 本章介绍线程池中的Callable和Future.Callable 和 Future 简介示例和源码分析(基于JDK1.7.0_40) 转载请注明出处:http://www.cnblogs.co ...
- Java多线程系列--“JUC线程池”02之 线程池原理(一)
概要 在上一章"Java多线程系列--“JUC线程池”01之 线程池架构"中,我们了解了线程池的架构.线程池的实现类是ThreadPoolExecutor类.本章,我们通过分析Th ...
随机推荐
- ArcGIS api for javascript——地图配置-滑动器的刻度线、方向、大小的改变
描述 本例展示了如果删除缩放等级滑动器的刻度线.通过设置esriConfig里的sliderLabel为null来实现: esriConfig.defaults.map.sliderLabel = n ...
- wxWidgets笔记_1_linux环境下wxwidgets的安装与配置
linux下wxwidgets环境的安装与配置 一.建立目标文件夹(自己定义) mkdir /opt/SCOTT 二.安装wxWidgets 1.wxWidgets版本号能够选用wxGTK-2.8.1 ...
- POJ 3265 DP
思路: f[i][j]表示前i天能做j道题 (是做 不是做完) if(f[i-1][k]) if(suma[j]-suma[k]+g[i-1][k]<=n) f[i][j]=1,g[i][j]= ...
- Gym - 100203H Highways 最小生成树
题意:平面上n个点修路,已经修好了m条,再修若干条使得点之间连通,求最小代价的方案. 思路:基本上是裸的最小生成树了,我这里存边直接存在multyset了,取的时候也比较方便,我本来就是这么考虑的,队 ...
- HIVE的几种优化
5 WAYS TO MAKE YOUR HIVE QUERIES RUN FASTER 今天看了一篇[文章] (http://zh.hortonworks.com/blog/5-ways-make-h ...
- 如何使用通用pe工具箱破解开机密码
下载最新版的通用pe工具箱将u盘制作成启动盘,接着重启连续按热键进入到bios系统下,设置u盘为第一启动,保存重启. 1.这时候会进入通用pe工具箱的选择界面,我们选择第八个“运行Windows登陆密 ...
- Android使用token维持登陆状态的方法
什么是token token(令牌)是一串唯一的字符串,通常由服务端生成,在注册完成时返回给客户端,用来标识此用户,客户端将此字符串存储在本地.在以后的网络请求时,客户端先查询本地的token,如果有 ...
- ES6学习基础
1.let和const 与var不同,新的变量声明方式带来了一些不一样的特性,其中最重要的两个特性就是提供了块级作用域与不再具备变量提升 { let a = 20; } console.log(a); ...
- modSecurity规则学习(五)——DDOS攻击检测
1.IP访问频率 SecAction phase:1,nolog,pass,setvar:IP.counter=+1 SecRule IP:UPDATE_RATE "@gt 10" ...
- Android加载大图不OOM
首先,我们试着往sdcard里放一张400k的图片,但是分辨率是2560*1600 布局简单 <?xml version="1.0" encoding="utf-8 ...