java文件断点上传
1,项目调研
因为需要研究下断点上传的问题。找了很久终于找到一个比较好的项目。
在GoogleCode上面,代码弄下来超级不方便,还是配置hosts才好,把代码重新上传到了github上面。
效果:
上传中,显示进度,时间,百分比。
点击【Pause】暂停,点击【Resume】继续。
2,代码分析
原始项目:
项目进行了封装使用最简单的方法实现了http的断点上传。
因为html5 里面有读取文件分割文件的类库,所以才可以支持断点上传,所以这个只能在html5 支持的浏览器上面展示。
同时,在js 和 java 同时使用 cr32 进行文件块的校验,保证数据上传正确。
代码在使用了最新的servlet 3.0 的api,使用了异步执行,监听等方法。
上传类UploadServlet
@Component("javaLargeFileUploaderServlet")
@WebServlet(name = "javaLargeFileUploaderServlet", urlPatterns = { "/javaLargeFileUploaderServlet" })
public class UploadServlet extends HttpRequestHandlerServlet
implements HttpRequestHandler {
private static final Logger log = LoggerFactory.getLogger(UploadServlet.class);
@Autowired
UploadProcessor uploadProcessor;
@Autowired
FileUploaderHelper fileUploaderHelper;
@Autowired
ExceptionCodeMappingHelper exceptionCodeMappingHelper;
@Autowired
Authorizer authorizer;
@Autowired
StaticStateIdentifierManager staticStateIdentifierManager;
@Override
public void handleRequest(HttpServletRequest request, HttpServletResponse response)
throws IOException {
log.trace("Handling request");
Serializable jsonObject = null;
try {
// extract the action from the request
UploadServletAction actionByParameterName =
UploadServletAction.valueOf(fileUploaderHelper.getParameterValue(request, UploadServletParameter.action));
// check authorization
checkAuthorization(request, actionByParameterName);
// then process the asked action
jsonObject = processAction(actionByParameterName, request);
// if something has to be written to the response
if (jsonObject != null) {
fileUploaderHelper.writeToResponse(jsonObject, response);
}
}
// If exception, write it
catch (Exception e) {
exceptionCodeMappingHelper.processException(e, response);
}
}
private void checkAuthorization(HttpServletRequest request, UploadServletAction actionByParameterName)
throws MissingParameterException, AuthorizationException {
// check authorization
// if its not get progress (because we do not really care about authorization for get
// progress and it uses an array of file ids)
if (!actionByParameterName.equals(UploadServletAction.getProgress)) {
// extract uuid
final String fileIdFieldValue = fileUploaderHelper.getParameterValue(request, UploadServletParameter.fileId, false);
// if this is init, the identifier is the one in parameter
UUID clientOrJobId;
String parameter = fileUploaderHelper.getParameterValue(request, UploadServletParameter.clientId, false);
if (actionByParameterName.equals(UploadServletAction.getConfig) && parameter != null) {
clientOrJobId = UUID.fromString(parameter);
}
// if not, get it from manager
else {
clientOrJobId = staticStateIdentifierManager.getIdentifier();
}
// call authorizer
authorizer.getAuthorization(
request,
actionByParameterName,
clientOrJobId,
fileIdFieldValue != null ? getFileIdsFromString(fileIdFieldValue).toArray(new UUID[] {}) : null);
}
}
private Serializable processAction(UploadServletAction actionByParameterName, HttpServletRequest request)
throws Exception {
log.debug("Processing action " + actionByParameterName.name());
Serializable returnObject = null;
switch (actionByParameterName) {
case getConfig:
String parameterValue = fileUploaderHelper.getParameterValue(request, UploadServletParameter.clientId, false);
returnObject =
uploadProcessor.getConfig(
parameterValue != null ? UUID.fromString(parameterValue) : null);
break;
case verifyCrcOfUncheckedPart:
returnObject = verifyCrcOfUncheckedPart(request);
break;
case prepareUpload:
returnObject = prepareUpload(request);
break;
case clearFile:
uploadProcessor.clearFile(UUID.fromString(fileUploaderHelper.getParameterValue(request, UploadServletParameter.fileId)));
break;
case clearAll:
uploadProcessor.clearAll();
break;
case pauseFile:
List<UUID> uuids = getFileIdsFromString(fileUploaderHelper.getParameterValue(request, UploadServletParameter.fileId));
uploadProcessor.pauseFile(uuids);
break;
case resumeFile:
returnObject =
uploadProcessor.resumeFile(UUID.fromString(fileUploaderHelper.getParameterValue(request, UploadServletParameter.fileId)));
break;
case setRate:
uploadProcessor.setUploadRate(UUID.fromString(fileUploaderHelper.getParameterValue(request, UploadServletParameter.fileId)),
Long.valueOf(fileUploaderHelper.getParameterValue(request, UploadServletParameter.rate)));
break;
case getProgress:
returnObject = getProgress(request);
break;
}
return returnObject;
}
List<UUID> getFileIdsFromString(String fileIds) {
String[] splittedFileIds = fileIds.split(",");
List<UUID> uuids = Lists.newArrayList();
for (int i = 0; i < splittedFileIds.length; i++) {
uuids.add(UUID.fromString(splittedFileIds[i]));
}
return uuids;
}
private Serializable getProgress(HttpServletRequest request)
throws MissingParameterException {
Serializable returnObject;
String[] ids =
new Gson()
.fromJson(fileUploaderHelper.getParameterValue(request, UploadServletParameter.fileId), String[].class);
Collection<UUID> uuids = Collections2.transform(Arrays.asList(ids), new Function<String, UUID>() {
@Override
public UUID apply(String input) {
return UUID.fromString(input);
}
});
returnObject = Maps.newHashMap();
for (UUID fileId : uuids) {
try {
ProgressJson progress = uploadProcessor.getProgress(fileId);
((HashMap<String, ProgressJson>) returnObject).put(fileId.toString(), progress);
}
catch (FileNotFoundException e) {
log.debug("No progress will be retrieved for " + fileId + " because " + e.getMessage());
}
}
return returnObject;
}
private Serializable prepareUpload(HttpServletRequest request)
throws MissingParameterException, IOException {
// extract file information
PrepareUploadJson[] fromJson =
new Gson()
.fromJson(fileUploaderHelper.getParameterValue(request, UploadServletParameter.newFiles), PrepareUploadJson[].class);
// prepare them
final HashMap<String, UUID> prepareUpload = uploadProcessor.prepareUpload(fromJson);
// return them
return Maps.newHashMap(Maps.transformValues(prepareUpload, new Function<UUID, String>() {
public String apply(UUID input) {
return input.toString();
};
}));
}
private Boolean verifyCrcOfUncheckedPart(HttpServletRequest request)
throws IOException, MissingParameterException, FileCorruptedException, FileStillProcessingException {
UUID fileId = UUID.fromString(fileUploaderHelper.getParameterValue(request, UploadServletParameter.fileId));
try {
uploadProcessor.verifyCrcOfUncheckedPart(fileId,
fileUploaderHelper.getParameterValue(request, UploadServletParameter.crc));
}
catch (InvalidCrcException e) {
// no need to log this exception, a fallback behaviour is defined in the
// throwing method.
// but we need to return something!
return Boolean.FALSE;
}
return Boolean.TRUE;
}
}
异步上传UploadServletAsync
@Component("javaLargeFileUploaderAsyncServlet")
@WebServlet(name = "javaLargeFileUploaderAsyncServlet", urlPatterns = { "/javaLargeFileUploaderAsyncServlet" }, asyncSupported = true)
public class UploadServletAsync extends HttpRequestHandlerServlet
implements HttpRequestHandler {
private static final Logger log = LoggerFactory.getLogger(UploadServletAsync.class);
@Autowired
ExceptionCodeMappingHelper exceptionCodeMappingHelper;
@Autowired
UploadServletAsyncProcessor uploadServletAsyncProcessor;
@Autowired
StaticStateIdentifierManager staticStateIdentifierManager;
@Autowired
StaticStateManager<StaticStatePersistedOnFileSystemEntity> staticStateManager;
@Autowired
FileUploaderHelper fileUploaderHelper;
@Autowired
Authorizer authorizer;
/**
* Maximum time that a streaming request can take.<br>
*/
private long taskTimeOut = DateUtils.MILLIS_PER_HOUR;
@Override
public void handleRequest(final HttpServletRequest request, final HttpServletResponse response)
throws ServletException, IOException {
// process the request
try {
//check if uploads are allowed
if (!uploadServletAsyncProcessor.isEnabled()) {
throw new UploadIsCurrentlyDisabled();
}
// extract stuff from request
final FileUploadConfiguration process = fileUploaderHelper.extractFileUploadConfiguration(request);
log.debug("received upload request with config: "+process);
// verify authorization
final UUID clientId = staticStateIdentifierManager.getIdentifier();
authorizer.getAuthorization(request, UploadServletAction.upload, clientId, process.getFileId());
//check if that file is not paused
if (uploadServletAsyncProcessor.isFilePaused(process.getFileId())) {
log.debug("file "+process.getFileId()+" is paused, ignoring async request.");
return;
}
// get the model
StaticFileState fileState = staticStateManager.getEntityIfPresent().getFileStates().get(process.getFileId());
if (fileState == null) {
throw new FileNotFoundException("File with id " + process.getFileId() + " not found");
}
// process the request asynchronously
final AsyncContext asyncContext = request.startAsync();
asyncContext.setTimeout(taskTimeOut);
// add a listener to clear bucket and close inputstream when process is complete or
// with
// error
asyncContext.addListener(new UploadServletAsyncListenerAdapter(process.getFileId()) {
@Override
void clean() {
log.debug("request " + request + " completed.");
// we do not need to clear the inputstream here.
// and tell processor to clean its shit!
uploadServletAsyncProcessor.clean(clientId, process.getFileId());
}
});
// then process
uploadServletAsyncProcessor.process(fileState, process.getFileId(), process.getCrc(), process.getInputStream(),
new WriteChunkCompletionListener() {
@Override
public void success() {
asyncContext.complete();
}
@Override
public void error(Exception exception) {
// handles a stream ended unexpectedly , it just means the user has
// stopped the
// stream
if (exception.getMessage() != null) {
if (exception.getMessage().equals("Stream ended unexpectedly")) {
log.warn("User has stopped streaming for file " + process.getFileId());
}
else if (exception.getMessage().equals("User cancellation")) {
log.warn("User has cancelled streaming for file id " + process.getFileId());
// do nothing
}
else {
exceptionCodeMappingHelper.processException(exception, response);
}
}
else {
exceptionCodeMappingHelper.processException(exception, response);
}
asyncContext.complete();
}
});
}
catch (Exception e) {
exceptionCodeMappingHelper.processException(e, response);
}
}
}
3,请求流程图:
主要思路就是将文件切分,然后分块上传。
详细代码可以参照我写的这篇文章:http://blog.ncmem.com/wordpress/2019/08/12/%e5%a4%a7%e6%96%87%e4%bb%b6%e6%96%ad%e7%82%b9%e7%bb%ad%e4%bc%a0-2/
java文件断点上传的更多相关文章
- 后端springmvc,前端html5的FormData实现文件断点上传
前言 最近项目中有使用到文件断点上传,得空便总结总结,顺便记录一下,毕竟“好记性不如烂笔头”. 后端代码: package com.test.controller; import java.io.Bu ...
- 文件断点上传,html5实现前端,java实现服务器
断点上传能够防止意外情况导致上传一半的文件下次上传时还要从头下载,网上有很多关于断点的实现,这篇文章只是从前到后完整的记录下一个可用的实例,由于生产环境要求不高,而且就是提供给一两个人用,所以我简化了 ...
- java HTTP文件断点上传
之前仿造uploadify写了一个HTML5版的文件上传插件,没看过的朋友可以点此先看一下~得到了不少朋友的好评,我自己也用在了项目中,不论是用户头像上传,还是各种媒体文件的上传,以及各种个性的业务需 ...
- Android中Socket大文件断点上传
原文:http://blog.csdn.net/shimiso/article/details/8529633 什么是Socket? 所谓Socket通常也称作“套接字”,用于描述IP地址和端口,是一 ...
- Android应用开发之使用Socket进行大文件断点上传续传
http://www.linuxidc.com/Linux/2012-03/55567.htm http://blog.csdn.net/shimiso/article/details/8529633 ...
- java 文件的上传和下载
主要介绍使用 smartupload.jar 包中的方法对文件的上传和下载.上传时文件是存放在服务器中,我用的是tamcat. 首先建立一个servlet 类,对文件的操作 package com.d ...
- java文件夹上传下载控件分享
用过浏览器的开发人员都对大文件上传与下载比较困扰,之前遇到了一个需要在JAVA.MyEclipse环境下大文件上传的问题,无奈之下自己开发了一套文件上传控件,在这里分享一下.希望能对你有所帮助. 以下 ...
- java文件分片上传,断点续传
百度的webUploader的前端开源插件实现的大文件分片上传功能 前端部分 前端页面代码如下,只需要修改自己的文件上传地址接口地址: <!DOCTYPE html> <html l ...
- java文件断点续传上传下载解决方案
这里只写后端的代码,基本的思想就是,前端将文件分片,然后每次访问上传接口的时候,向后端传入参数:当前为第几块文件,和分片总数 下面直接贴代码吧,一些难懂的我大部分都加上注释了: 上传文件实体类: 看得 ...
随机推荐
- springmvc 读写分离
推荐第四种:https://github.com/shawntime/shawn-rwdb 4种不方的读写分离实现方法 http://blog.csdn.net/lixiucheng005/artic ...
- VMware vMotion 配置要求
目录 目录 vCenter 支持 vMotion 的前提 条件 vMotion 的主机配置 vMotion 共享存储器要求 vMotion 网络要求 最后 vCenter 支持 vMotion 的前提 ...
- MM理论
最初的MM理论,即由美国的Modigliani和Miller(简称MM)教授于1958年6月份发表于<美国经济评论>的“资本结构.公司财务与资本”一文中所阐述的基本思想.该理论认为,在不考 ...
- Selenium WebDriver 数据驱动测试框架
Selenium WebDriver 数据驱动测试框架,以QQ邮箱添加联系人为示例,测试框架结构如下图,详细内容请阅读吴晓华编著<Selenium WebDiver 实战宝典>: Obje ...
- 神器 工具 推荐 SRDebugger
unity asset store 关联下载 ,添加这个书签 javascript:var url = window.location.href;var id = url.substr(url.la ...
- TNS-01106: Listener using listener name LISTENER has already been started
-- 启动监听,提示已经启动. [oracle@sh ~]$ lsnrctl startLSNRCTL for Linux: Version 12.1.0.2.0 - Production on 06 ...
- VLAN 基础设置及Aceess接口
实验内容 本实验模拟企业网络场景.公司内网是一一个大的局域网,二层交换机S1放置在一楼,在一楼办公的部门有IT部和人事部;二层交换机S2放置在二楼,在二楼办公的部门有市场部和研发部.由于交换机组成的是 ...
- Java中的常用类:包装类、String、StringBuffer、StringBuilder、Math、System、Arrays、BigInteger、BigDecimal、Data、Calendar
一.包装类 √ 二.String类 ★ 三.StringBuffer和StringBuilder类 ★ 四.Math类 五.System类 六.Arrays类 七.BigInteger类和BigDec ...
- Java可变参数方法
概念: jdk5.0出现的新特性.将同一个类中,多个方法名相同.参数类型相同.返回类型相同,仅仅是参数个数不同的方法抽取成一个方法,这种方法称为可变参数的方法 好处: 提高代码的重用性和维护性 语法: ...
- 本地启oracle实例服务无法重启,协议适配器错误
今天遇到一位朋友的oracle实例服务无法起来,启动时报错: 分析的原因是可能早上服务器突然断电造成的,经过对tns的测试 经过我们讨论和诊断,最后诊断的处理方法是将实例删了重装,处理后服务恢复正常: ...