Android 多线程下载,断点续传,线程池
你可以在这里看到这个demo的源码:
https://github.com/onlynight/MultiThreadDownloader
效果图
这张效果图是同时开启三个下载任务,限制下载线程数量的效果图。
多线程下载原理
多线程下载的原理就是将下载任务分割成一个个小片段再将每个小片段分配给各个线程进行下载。
例如一个文件大小为100M,我们决定使用4个线程下载,那么每个线程下载的大小即为25M,每个线程的起始以及结束位置依次如下:
0: 0-25M
1: 25-50M
2: 50-75M
3: 75-END
下载请求使用的是Http GET方法,GET请求允许我们设置参数请求的数据的范围:
HttpURLConnection conn = (HttpURLConnection)
new URL("download url").openConnection();
// 设置http为get方法
conn.setRequestMethod("GET");
// 设置http请求的范围
conn.setRequestProperty("Range", "bytes=" + startPos + "-" + endPos);
多线程访问文件需要创建RandomAccessFile:
RandomAccessFile threadFile = new RandomAccessFile(
fileDownloader.getFileSavePath() + File.separator +
fileDownloader.getFilename(), "rwd");
threadFile.seek(startPos);
断点续传原理
端点续传要求我们将之前下载过的文件片段保存下来,并且记录上次最后的下载位置。下次继续下载的时候从上次记录的位置开始下载即可,无需再从头下载。
线程池作用
线程池的原理分析和使用请查看者几篇文章:
http://blog.csdn.net/u010687392/article/details/49850803
http://blog.csdn.net/qq_17250009/article/details/50906508
试想如果我们有100个下载任务,我们让每个任务分成3个线程下载,那么每个任务都需要4个线程,如果100个任务同时开启下载那么就意味着需要同时启动400个线程执行下载任务,这样势必会影响app性能。
像上面这样大量的创建线程的操作势必会影响操作系统的性能,等这些任务执行完成后销毁线程同样也会消耗很多的系统资源,所以Java中提出了线程池的概念。
下面我们来分析线程池是如何节约系统资源的。
从时间角度:
- 创线程的时间我们假定它为: t1
- 线程执行的时间我们假定为: t2
- 销毁线程的时间我们假定为: t3
就用上例我们计算其耗时为:
T = 100(t1+t2+t3)
我们使用固定上限线程数量的线程池耗时为:
T1 = 5(t1+t3)+100t2
显然固定线程上限线程池所需的时间短了很多,固定数量线程池节约了线程创建和销毁的时间,使用线程复用方法避免了线程的频繁创建和销毁,不仅节约了时间同时节约了系统资源。
关键代码
下载线程:
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.io.RandomAccessFile;
import java.net.HttpURLConnection;
import java.net.URL; /**
* Created by lion on 2017/2/10.
*/ public class DownloadRunnable implements Runnable {
private int threadId = -;
private FileDownloader fileDownloader;
private int downloadedSize = ; private int startPos = -;
private int endPos = -;
private int downloadLength = ; private boolean isFinish;
private boolean isStart; public DownloadRunnable(FileDownloader fileDownloader, int threadId, int blockSize,
int downloadedSize) {
this.fileDownloader = fileDownloader;
this.threadId = threadId;
int fileSize = fileDownloader.getFileSize();
this.startPos = blockSize * threadId + downloadedSize;
this.endPos = blockSize * (threadId + ) < fileSize ?
blockSize * (threadId + ) : fileSize;
this.downloadedSize = downloadedSize;
} @Override
public void run() {
if (startPos >= endPos) {
isFinish = true;
} else {
try {
isStart = true;
isFinish = false;
HttpURLConnection conn = (HttpURLConnection)
new URL(fileDownloader.getDownloadUrl()).openConnection();
conn.setConnectTimeout(FileDownloader.getConnectionTimeOut());
conn.setRequestMethod("GET"); //set accept file meta-data type
conn.setRequestProperty("Accept", "image/gif, image/jpeg, image/pjpeg," +
" image/pjpeg, application/x-shockwave-flash, application/xaml+xml, " +
"application/vnd.ms-xpsdocument, application/x-ms-xbap, " +
"application/x-ms-application, application/vnd.ms-excel, " +
"application/vnd.ms-powerpoint, application/msword, */*"); conn.setRequestProperty("Accept-Language", "zh-CN");
conn.setRequestProperty("Referer", fileDownloader.getDownloadUrl());
conn.setRequestProperty("Charset", "UTF-8");
conn.setRequestProperty("User-Agent", "Mozilla/4.0 (compatible; MSIE 8.0; " +
"Windows NT 5.2; Trident/4.0; .NET CLR 1.1.4322; .NET CLR 2.0.50727; " +
".NET CLR 3.0.04506.30; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)");
conn.setRequestProperty("Connection", "Keep-Alive"); conn.setRequestProperty("Range", "bytes=" + startPos + "-" + endPos);
conn.connect(); RandomAccessFile threadFile = new RandomAccessFile(
fileDownloader.getFileSavePath() + File.separator +
fileDownloader.getFilename(), "rwd");
threadFile.seek(startPos);
InputStream inputStream = conn.getInputStream();
byte[] buffer = new byte[];
int offset;
downloadLength = downloadedSize;
while ((offset = inputStream.read(buffer, , )) != -) {
threadFile.write(buffer, , offset);
downloadLength += offset;
fileDownloader.appendDownloadSize(offset);
}
threadFile.close();
inputStream.close(); isFinish = true;
isStart = false;
} catch (IOException e) {
e.printStackTrace();
downloadLength = -;
}
}
} public int getDownloadLength() {
return downloadLength;
} public int getThreadId() {
return threadId;
} public boolean isFinish() {
return isFinish;
} public boolean isStart() {
return isStart;
}
}
下载器:
import android.content.Context;
import android.util.SparseIntArray; import java.io.IOException;
import java.net.HttpURLConnection;
import java.net.URL;
import java.util.Date;
import java.util.UUID;
import java.util.regex.Matcher;
import java.util.regex.Pattern; /**
* Created by lion on 2017/2/7.
*/ public class FileDownloader { public static final String TAG = "FileDownloader"; /**
* http connection timeout
*/
private static int CONNECTION_TIME_OUT = * ; private DownloadProgressManager downloadProgressManager; private DownloadRunnable[] downloadThreads; private String tagName = ""; private String downloadUrl;
private String fileSavePath;
private String filename;
private int threadNum = ;
private int fileSize = ;
private int currentDownloadSize = ; private SparseIntArray currentDownloads; public DownloadRunnable[] getDownloadThreads() {
return downloadThreads;
} public DownloadProgressManager getDownloadProgressManager() {
return downloadProgressManager;
} public int getFileSize() {
return fileSize;
} public String getFileSavePath() {
return fileSavePath;
} public String getFilename() {
return filename;
} public String getDownloadUrl() {
return downloadUrl;
} public int getCurrentDownloadSize() {
return currentDownloadSize;
} public int getThreadNum() {
return threadNum;
} synchronized int appendDownloadSize(int size) {
currentDownloadSize += size;
return currentDownloadSize;
} public FileDownloader(Context context) {
this.currentDownloads = new SparseIntArray();
this.downloadProgressManager = new DownloadProgressManager(context);
} private void requestFileInfo(String downloadUrl) throws RuntimeException {
try {
HttpURLConnection connection = (HttpURLConnection)
new URL(downloadUrl).openConnection();
connection.setConnectTimeout(CONNECTION_TIME_OUT);
connection.setRequestMethod("GET"); //set accept file meta-data type
connection.setRequestProperty("Accept", "image/gif, image/jpeg, image/pjpeg," +
" image/pjpeg, application/x-shockwave-flash, application/xaml+xml, " +
"application/vnd.ms-xpsdocument, application/x-ms-xbap, " +
"application/x-ms-application, application/vnd.ms-excel, " +
"application/vnd.ms-powerpoint, application/msword, */*"); connection.setRequestProperty("Accept-Language", "zh-CN");
connection.setRequestProperty("Referer", downloadUrl);
connection.setRequestProperty("Charset", "UTF-8");
connection.setRequestProperty("User-Agent", "Mozilla/4.0 (compatible; MSIE 8.0; " +
"Windows NT 5.2; Trident/4.0; .NET CLR 1.1.4322; .NET CLR 2.0.50727; " +
".NET CLR 3.0.04506.30; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)");
// connection.setRequestProperty("Connection", "Keep-Alive"); connection.connect(); if (connection.getResponseCode() == ) {
fileSize = connection.getContentLength();
if (fileSize <= ) {
throw new RuntimeException(TAG + " Unknown file size");
} filename = getFilename(connection);
} else {
throw new RuntimeException(TAG + " Server Response Code is "
+ connection.getResponseCode());
}
} catch (IOException e) {
e.printStackTrace();
}
} private String getFilename(HttpURLConnection connection) {
String filename = downloadUrl != null ?
downloadUrl.substring(downloadUrl.lastIndexOf("/") + ) : null;
if (filename == null || "".equals(filename.trim())) {//如果获取不到文件名称
for (int i = ; ; i++) {
String mine = connection.getHeaderField(i);
if (mine == null) break;
if ("content-disposition".equals(connection.getHeaderFieldKey(i).toLowerCase())) {
Matcher m = Pattern.compile(".*filename=(.*)").
matcher(mine.toLowerCase());
if (m.find()) return m.group();
}
}
filename = UUID.randomUUID() + ".tmp";//默认取一个文件名
}
return filename;
} public void prepare(String downloadUrl, String fileSavePath, int threadNum) {
this.downloadUrl = downloadUrl;
this.fileSavePath = fileSavePath;
requestFileInfo(downloadUrl);
SparseIntArray progresses = downloadProgressManager.getProgress(downloadUrl); if (threadNum <= ) {
threadNum = this.threadNum;
} else {
this.threadNum = threadNum;
} if (progresses != null && progresses.size() > ) {
threadNum = progresses.size();
for (int i = ; i < progresses.size(); i++) {
currentDownloadSize += progresses.get(i);
}
} int block = fileSize % threadNum == ?
fileSize / threadNum : fileSize / threadNum + ; downloadThreads = new DownloadRunnable[threadNum]; for (int i = ; i < threadNum; i++) {
downloadThreads[i] = new DownloadRunnable(this, i, block,
progresses != null && progresses.size() == threadNum ?
progresses.valueAt(progresses.keyAt(i)) == - ? :
progresses.valueAt(progresses.keyAt(i)) : );
}
} public void start(OnDownloadListener listener) {
boolean isFinish = false;
int lastDownloadSize = ;
int speed = ;
Date current = new Date();
while (!isFinish) {
if (listener != null) {
int percent = (int) (currentDownloadSize / (float) fileSize * );
long time = new Date().getTime() - current.getTime();
speed = (int) ((currentDownloadSize - lastDownloadSize) / 1024f / time * 1000f);
listener.onUpdate(fileSize, currentDownloadSize, speed, percent);
if (percent == ) {
downloadProgressManager.finishDownload(downloadUrl);
break;
}
}
current = new Date();
lastDownloadSize = currentDownloadSize;
updateProgress();
isFinish = checkFinish();
try {
Thread.sleep();
} catch (InterruptedException e) {
e.printStackTrace();
}
} // System.out.println(tagName + " DOWNLOAD FINISH");
if (listener != null) {
listener.onUpdate(fileSize, fileSize, , );
}
} private boolean checkFinish() {
if (downloadThreads != null && downloadThreads.length > ) {
for (DownloadRunnable downloadThread : downloadThreads) {
if (!downloadThread.isFinish()) {
System.out.println("checkFinish false");
return false;
}
} return true;
}
System.out.println("checkFinish true");
return false;
} public boolean isFinish() {
return checkFinish();
} void updateProgress() {
for (DownloadRunnable downloadThread : downloadThreads) {
updateProgress(downloadThread.getThreadId(), downloadThread.getDownloadLength());
}
} synchronized void updateProgress(int threadId, int downloaded) {
currentDownloads.put(threadId, downloaded);
downloadProgressManager.saveProgress(downloadUrl, currentDownloads);
// SparseIntArray progress = downloadProgressManager.getProgress(downloadUrl);
// for (int i = 0; i < progress.size(); i++) {
// System.out.println("prepare progress = " + progress.valueAt(progress.keyAt(i)));
// }
} public boolean isStart() {
for (DownloadRunnable runnable : downloadThreads) {
if (runnable.isStart()) {
return true;
}
} return false;
} static int getConnectionTimeOut() {
return CONNECTION_TIME_OUT;
} static void setConnectionTimeOut(int timeOut) {
CONNECTION_TIME_OUT = timeOut;
} public interface OnDownloadListener {
void onUpdate(int totalSize, int currentSize, int speed, int percent);
} public void setTagName(String tagName) {
this.tagName = tagName;
}
}
下载管理器:
import android.content.Context; import java.util.ArrayList;
import java.util.concurrent.Executor;
import java.util.concurrent.Executors; /**
* Created by lion on 2017/2/8.
*/ public class DownloadManager { private static int PARALLEL_DOWNLOAD_SIZE = ;
private static DownloadManager instance; private Context context;
private Executor downloadExecutor;
private ArrayList<FileDownloader> fileDownloaders; public static DownloadManager getInstance(Context context) {
if (instance == null) {
instance = new DownloadManager(context);
}
return instance;
} public DownloadManager(Context context) {
this.context = context;
downloadExecutor = Executors.newFixedThreadPool(PARALLEL_DOWNLOAD_SIZE);
// downloadExecutor = Executors.newCachedThreadPool();
fileDownloaders = new ArrayList<>();
} public void download(String name, final String downloadUrl, final String fileSavePath, final int threadNum,
final FileDownloader.OnDownloadListener listener) {
for (FileDownloader downloader : fileDownloaders) {
if (downloader.isFinish()) {
downloader.setTagName(name);
startDownload(downloader, downloadUrl, fileSavePath, threadNum, listener);
return;
}
} FileDownloader currentDownloader = new FileDownloader(context);
currentDownloader.setTagName(name);
fileDownloaders.add(currentDownloader);
startDownload(currentDownloader, downloadUrl, fileSavePath, threadNum, listener);
} public void download(final String downloadUrl, final String fileSavePath, final int threadNum,
final FileDownloader.OnDownloadListener listener) {
for (FileDownloader downloader : fileDownloaders) {
if (downloader.isFinish()) {
startDownload(downloader, downloadUrl, fileSavePath, threadNum, listener);
return;
}
} FileDownloader currentDownloader = new FileDownloader(context);
fileDownloaders.add(currentDownloader);
startDownload(currentDownloader, downloadUrl, fileSavePath, threadNum, listener);
} private synchronized void startDownload(final FileDownloader currentDownloader,
final String downloadUrl, final String fileSavePath,
final int threadNum,
final FileDownloader.OnDownloadListener listener) {
downloadExecutor.execute(new Runnable() {
@Override
public void run() {
currentDownloader.prepare(downloadUrl, fileSavePath,
threadNum);
if (currentDownloader.getDownloadThreads() != null) {
for (DownloadRunnable runnable :
currentDownloader.getDownloadThreads()) {
downloadExecutor.execute(runnable);
}
}
currentDownloader.start(listener);
}
});
} public static void setConnectionTimeOut(int timeOut) {
FileDownloader.setConnectionTimeOut(timeOut);
} public static void setParallelDownloadSize(int size) {
PARALLEL_DOWNLOAD_SIZE = size;
}
}
Android 多线程下载,断点续传,线程池的更多相关文章
- android 多线程下载 断点续传
来源:网易云课堂Android极客班第八次作业练习 练习内容: 多线程 asyncTask handler 多线程下载的原理 首先获取到目标文件的大小,然后在磁盘上申请一块空间用于保存目标文件,接着把 ...
- 【多线程】Android多线程学习笔记——线程池
Java线程池采用了享元设计模式,在系统中维持一定数量的线程,用于处理异步或并发需求,在平时处理异步或并发任务时被广泛使用.这里基于JDK1.8和Android28来整理一些关于线程池的知识点. 一. ...
- android程序---->android多线程下载(一)
多线程下载是加快下载速度的一种方式,通过开启多个线程去执行一个任务,可以使任务的执行速度变快.多线程的任务下载时常都会使用得到断点续传下载,就是我们在一次下载未结束时退出下载,第二次下载时会接着第一次 ...
- android程序---->android多线程下载(二)
上篇我们讲到了android中下载的断点续传问题,今天我们开始学习下载的多线程问题.本次的多线程源码下载:androdi中多线程下载的实现代码.有关断点续传的问题,请参见博客:android程序--- ...
- 无废话Android之smartimageview使用、android多线程下载、显式意图激活另外一个activity,检查网络是否可用定位到网络的位置、隐式意图激活另外一个activity、隐式意图的配置,自定义隐式意图、在不同activity之间数据传递(5)
1.smartimageview使用 <LinearLayout xmlns:android="http://schemas.android.com/apk/res/android&q ...
- Andoid 更好的Android多线程下载框架
概述 为什么是更好的Android多线程下载框架呢,原因你懂的,广告法嘛! 本篇我们我们就来聊聊多线程下载框架,先聊聊我们框架的特点: 多线程 多任务 断点续传 支持大文件 可以自定义下载数据库 高度 ...
- 更好的Android多线程下载框架
/** * 作者:Pich * 原文链接:http://me.woblog.cn/ * QQ群:129961195 * Github:https://github.com/lifengsofts */ ...
- Java多线程系列--“JUC线程池”06之 Callable和Future
概要 本章介绍线程池中的Callable和Future.Callable 和 Future 简介示例和源码分析(基于JDK1.7.0_40) 转载请注明出处:http://www.cnblogs.co ...
- Java多线程系列--“JUC线程池”02之 线程池原理(一)
概要 在上一章"Java多线程系列--“JUC线程池”01之 线程池架构"中,我们了解了线程池的架构.线程池的实现类是ThreadPoolExecutor类.本章,我们通过分析Th ...
随机推荐
- Dojo入门篇
Dojo是一个JavaScript实现的开源DHTML工具包,Dojo最初的目标是解决开发HTML应用程序中遇到的一些长期存在的问题.然而如今Dojo已经成为了开发RIA应用程序的利器. Dojo让W ...
- Android设计模式(十二)--抽象工厂模式
问题: 抽象工厂模式,是一个,狠恶心的模式,那么这个模式在Android有没实用到过呢? 1.定义: 抽象工厂模式:为创建一组相关或者是相互依赖的对象提供一个接口,而不须要指定他们的详细类. 2.使用 ...
- poj1014 hdu1059 Dividing 多重背包
有价值为1~6的宝物各num[i]个,求能否分成价值相等的两部分. #include <iostream> #include <cstring> #include <st ...
- Android 数据库框架总结,总有一个适合你!
一:OrmLite 简述: 优点: 1.轻量级:2.使用简单,易上手:3.封装完善:4.文档全面.缺点:1.基于反射,效率较低(本人还没有觉得效率低):2.缺少中文翻译文档 jar包 地址:http: ...
- .net开源CMS
提起开源cms,大家第一想到的是php的cms,因为php开源的最早,也最为用户和站长们认可,随着各大cms系统的功能的不断完善和各式各样的开源cms的出现,.net和java的高端的cms系统也逐渐 ...
- 学习《R数据科学》高清中文PDF+高清英文PDF+源代码
学习R有不会的就查工具书<R数据科学>, 工具不是重点,创造价值才是目的.具体到数据科学,表现形式往往是提供解决方案或者做出某种决策.至于使用什么语言,采用什么工具,不本质.用 R 还是 ...
- Swift学习笔记(15)--下标脚本(Subscripts)
下标脚本可以定义在类(Class).结构体(structure)和枚举(enumeration)这些目标中,使用中类似数组或者字典的用法 1.定义 定义下标脚本使用subscript关键字,语法: s ...
- 洛谷——P1043 数字游戏
https://www.luogu.org/problem/show?pid=1043 题目描述 丁丁最近沉迷于一个数字游戏之中.这个游戏看似简单,但丁丁在研究了许多天之后却发觉原来在简单的规则下想要 ...
- android ActionBar的使用
Action Bar主要功能包括: 1. 显示选项菜单 2. 提供标签页的切换方式的导航功能,能够切换多个fragment. 3. 提供下拉的导航条目. 4. 提供交互式活动视图取 ...
- iOS使用push隐藏子页面底部bottom TabBar
下面两种情况是我在开发过程中遇到的,一种是代码使用pushViewController,还有一种是storyboard直接使用push.之前也查阅了非常多关于隐藏底部tabbar的资料.可是要么使用起 ...