1,项目调研

因为需要研究下断点上传的问题。找了很久终于找到一个比较好的项目。

在GoogleCode上面,代码弄下来超级不方便,还是配置hosts才好,把代码重新上传到了github上面。

https://github.com/freewebsys/java-large-file-uploader-demo

效果:

上传中,显示进度,时间,百分比。

点击【Pause】暂停,点击【Resume】继续。

2,代码分析

原始项目:

https://code.google.com/p/java-large-file-uploader/

这个项目最后更新的时间是 2012 年,项目进行了封装使用最简单的方法实现了http的断点上传。

因为html5 里面有读取文件分割文件的类库,所以才可以支持断点上传,所以这个只能在html5 支持的浏览器上面展示。

同时,在js 和 java 同时使用 cr32 进行文件块的校验,保证数据上传正确。

代码在使用了最新的servlet 3.0 的api,使用了异步执行,监听等方法。

上传类UploadServlet

@Component("javaLargeFileUploaderServlet")

@WebServlet(name = "javaLargeFileUploaderServlet", urlPatterns = { "/javaLargeFileUploaderServlet" })

public class UploadServlet extends HttpRequestHandlerServlet

implements HttpRequestHandler {

private static final Logger log = LoggerFactory.getLogger(UploadServlet.class);

@Autowired

UploadProcessor uploadProcessor;

@Autowired

FileUploaderHelper fileUploaderHelper;

@Autowired

ExceptionCodeMappingHelper exceptionCodeMappingHelper;

@Autowired

Authorizer authorizer;

@Autowired

StaticStateIdentifierManager staticStateIdentifierManager;

@Override

public void handleRequest(HttpServletRequest request, HttpServletResponse response)

throws IOException {

log.trace("Handling request");

Serializable jsonObject = null;

try {

// extract the action from the request

UploadServletAction actionByParameterName =

UploadServletAction.valueOf(fileUploaderHelper.getParameterValue(request, UploadServletParameter.action));

// check authorization

checkAuthorization(request, actionByParameterName);

// then process the asked action

jsonObject = processAction(actionByParameterName, request);

// if something has to be written to the response

if (jsonObject != null) {

fileUploaderHelper.writeToResponse(jsonObject, response);

}

}

// If exception, write it

catch (Exception e) {

exceptionCodeMappingHelper.processException(e, response);

}

}

private void checkAuthorization(HttpServletRequest request, UploadServletAction actionByParameterName)

throws MissingParameterException, AuthorizationException {

// check authorization

// if its not get progress (because we do not really care about authorization for get

// progress and it uses an array of file ids)

if (!actionByParameterName.equals(UploadServletAction.getProgress)) {

// extract uuid

final String fileIdFieldValue = fileUploaderHelper.getParameterValue(request, UploadServletParameter.fileId, false);

// if this is init, the identifier is the one in parameter

UUID clientOrJobId;

String parameter = fileUploaderHelper.getParameterValue(request, UploadServletParameter.clientId, false);

if (actionByParameterName.equals(UploadServletAction.getConfig) && parameter != null) {

clientOrJobId = UUID.fromString(parameter);

}

// if not, get it from manager

else {

clientOrJobId = staticStateIdentifierManager.getIdentifier();

}

// call authorizer

authorizer.getAuthorization(

request,

actionByParameterName,

clientOrJobId,

fileIdFieldValue != null ? getFileIdsFromString(fileIdFieldValue).toArray(new UUID[] {}) : null);

}

}

private Serializable processAction(UploadServletAction actionByParameterName, HttpServletRequest request)

throws Exception {

log.debug("Processing action " + actionByParameterName.name());

Serializable returnObject = null;

switch (actionByParameterName) {

case getConfig:

String parameterValue = fileUploaderHelper.getParameterValue(request, UploadServletParameter.clientId, false);

returnObject =

uploadProcessor.getConfig(

parameterValue != null ? UUID.fromString(parameterValue) : null);

break;

case verifyCrcOfUncheckedPart:

returnObject = verifyCrcOfUncheckedPart(request);

break;

case prepareUpload:

returnObject = prepareUpload(request);

break;

case clearFile:

uploadProcessor.clearFile(UUID.fromString(fileUploaderHelper.getParameterValue(request, UploadServletParameter.fileId)));

break;

case clearAll:

uploadProcessor.clearAll();

break;

case pauseFile:

List<UUID> uuids = getFileIdsFromString(fileUploaderHelper.getParameterValue(request, UploadServletParameter.fileId));

uploadProcessor.pauseFile(uuids);

break;

case resumeFile:

returnObject =

uploadProcessor.resumeFile(UUID.fromString(fileUploaderHelper.getParameterValue(request, UploadServletParameter.fileId)));

break;

case setRate:

uploadProcessor.setUploadRate(UUID.fromString(fileUploaderHelper.getParameterValue(request, UploadServletParameter.fileId)),

Long.valueOf(fileUploaderHelper.getParameterValue(request, UploadServletParameter.rate)));

break;

case getProgress:

returnObject = getProgress(request);

break;

}

return returnObject;

}

List<UUID> getFileIdsFromString(String fileIds) {

String[] splittedFileIds = fileIds.split(",");

List<UUID> uuids = Lists.newArrayList();

for (int i = 0; i < splittedFileIds.length; i++) {

uuids.add(UUID.fromString(splittedFileIds[i]));

}

return uuids;

}

private Serializable getProgress(HttpServletRequest request)

throws MissingParameterException {

Serializable returnObject;

String[] ids =

new Gson()

.fromJson(fileUploaderHelper.getParameterValue(request, UploadServletParameter.fileId), String[].class);

Collection<UUID> uuids = Collections2.transform(Arrays.asList(ids), new Function<String, UUID>() {

@Override

public UUID apply(String input) {

return UUID.fromString(input);

}

});

returnObject = Maps.newHashMap();

for (UUID fileId : uuids) {

try {

ProgressJson progress = uploadProcessor.getProgress(fileId);

((HashMap<String, ProgressJson>) returnObject).put(fileId.toString(), progress);

}

catch (FileNotFoundException e) {

log.debug("No progress will be retrieved for " + fileId + " because " + e.getMessage());

}

}

return returnObject;

}

private Serializable prepareUpload(HttpServletRequest request)

throws MissingParameterException, IOException {

// extract file information

PrepareUploadJson[] fromJson =

new Gson()

.fromJson(fileUploaderHelper.getParameterValue(request, UploadServletParameter.newFiles), PrepareUploadJson[].class);

// prepare them

final HashMap<String, UUID> prepareUpload = uploadProcessor.prepareUpload(fromJson);

// return them

return Maps.newHashMap(Maps.transformValues(prepareUpload, new Function<UUID, String>() {

public String apply(UUID input) {

return input.toString();

};

}));

}

private Boolean verifyCrcOfUncheckedPart(HttpServletRequest request)

throws IOException, MissingParameterException, FileCorruptedException, FileStillProcessingException {

UUID fileId = UUID.fromString(fileUploaderHelper.getParameterValue(request, UploadServletParameter.fileId));

try {

uploadProcessor.verifyCrcOfUncheckedPart(fileId,

fileUploaderHelper.getParameterValue(request, UploadServletParameter.crc));

}

catch (InvalidCrcException e) {

// no need to log this exception, a fallback behaviour is defined in the

// throwing method.

// but we need to return something!

return Boolean.FALSE;

}

return Boolean.TRUE;

}

}

异步上传UploadServletAsync

@Component("javaLargeFileUploaderAsyncServlet")

@WebServlet(name = "javaLargeFileUploaderAsyncServlet", urlPatterns = { "/javaLargeFileUploaderAsyncServlet" }, asyncSupported = true)

public class UploadServletAsync extends HttpRequestHandlerServlet

implements HttpRequestHandler {

private static final Logger log = LoggerFactory.getLogger(UploadServletAsync.class);

@Autowired

ExceptionCodeMappingHelper exceptionCodeMappingHelper;

@Autowired

UploadServletAsyncProcessor uploadServletAsyncProcessor;

@Autowired

StaticStateIdentifierManager staticStateIdentifierManager;

@Autowired

StaticStateManager<StaticStatePersistedOnFileSystemEntity> staticStateManager;

@Autowired

FileUploaderHelper fileUploaderHelper;

@Autowired

Authorizer authorizer;

/**

* Maximum time that a streaming request can take.<br>

*/

private long taskTimeOut = DateUtils.MILLIS_PER_HOUR;

@Override

public void handleRequest(final HttpServletRequest request, final HttpServletResponse response)

throws ServletException, IOException {

// process the request

try {

//check if uploads are allowed

if (!uploadServletAsyncProcessor.isEnabled()) {

throw new UploadIsCurrentlyDisabled();

}

// extract stuff from request

final FileUploadConfiguration process = fileUploaderHelper.extractFileUploadConfiguration(request);

log.debug("received upload request with config: "+process);

// verify authorization

final UUID clientId = staticStateIdentifierManager.getIdentifier();

authorizer.getAuthorization(request, UploadServletAction.upload, clientId, process.getFileId());

//check if that file is not paused

if (uploadServletAsyncProcessor.isFilePaused(process.getFileId())) {

log.debug("file "+process.getFileId()+" is paused, ignoring async request.");

return;

}

// get the model

StaticFileState fileState = staticStateManager.getEntityIfPresent().getFileStates().get(process.getFileId());

if (fileState == null) {

throw new FileNotFoundException("File with id " + process.getFileId() + " not found");

}

// process the request asynchronously

final AsyncContext asyncContext = request.startAsync();

asyncContext.setTimeout(taskTimeOut);

// add a listener to clear bucket and close inputstream when process is complete or

// with

// error

asyncContext.addListener(new UploadServletAsyncListenerAdapter(process.getFileId()) {

@Override

void clean() {

log.debug("request " + request + " completed.");

// we do not need to clear the inputstream here.

// and tell processor to clean its shit!

uploadServletAsyncProcessor.clean(clientId, process.getFileId());

}

});

// then process

uploadServletAsyncProcessor.process(fileState, process.getFileId(), process.getCrc(), process.getInputStream(),

new WriteChunkCompletionListener() {

@Override

public void success() {

asyncContext.complete();

}

@Override

public void error(Exception exception) {

// handles a stream ended unexpectedly , it just means the user has

// stopped the

// stream

if (exception.getMessage() != null) {

if (exception.getMessage().equals("Stream ended unexpectedly")) {

log.warn("User has stopped streaming for file " + process.getFileId());

}

else if (exception.getMessage().equals("User cancellation")) {

log.warn("User has cancelled streaming for file id " + process.getFileId());

// do nothing

}

else {

exceptionCodeMappingHelper.processException(exception, response);

}

}

else {

exceptionCodeMappingHelper.processException(exception, response);

}

asyncContext.complete();

}

});

}

catch (Exception e) {

exceptionCodeMappingHelper.processException(e, response);

}

}

}

3,请求流程图:

主要思路就是将文件切分,然后分块上传。

详细代码可以参照我写的这篇文章:http://t.cn/AiHwnJaB

jsp上传超大文件解决方案的更多相关文章

  1. js上传超大文件解决方案

    需求: 支持大文件批量上传(20G)和下载,同时需要保证上传期间用户电脑不出现卡死等体验: 内网百兆网络上传速度为12MB/S 服务器内存占用低 支持文件夹上传,文件夹中的文件数量达到1万个以上,且包 ...

  2. Web上传超大文件解决方案

    文件上传下载,与传统的方式不同,这里能够上传和下载10G以上的文件.而且支持断点续传. 通常情况下,我们在网站上面下载的时候都是单个文件下载,但是在实际的业务场景中,我们经常会遇到客户需要批量下载的场 ...

  3. B/S上传超大文件解决方案

    4GB以上超大文件上传和断点续传服务器的实现 随着视频网站和大数据应用的普及,特别是高清视频和4K视频应用的到来,超大文件上传已经成为了日常的基础应用需求. 但是在很多情况下,平台运营方并没有大文件上 ...

  4. java上传超大文件解决方案

    用JAVA实现大文件上传及显示进度信息 ---解析HTTP MultiPart协议 (本文提供全部源码下载,请访问 https://github.com/1269085759/up6-jsp-mysq ...

  5. .net上传超大文件解决方案

    HTML部分 <%@PageLanguage="C#"AutoEventWireup="true"CodeBehind="index.aspx. ...

  6. PHP上传超大文件解决方案

    一提到大文件上传,首先想到的是啥??? 没错,就是修改php.ini文件里的上传限制,那就是upload_max_filesize.修改成合适参数我们就可以进行愉快的上传文件了.当然啦,这是一般情况下 ...

  7. HTML5上传超大文件解决方案

    一.概述 所谓断点续传,其实只是指下载,也就是要从文件已经下载的地方开始继续下载.在以前版本的HTTP协议是不支持断点的,HTTP/1.1开始就支持了.一般断点下载时才用到Range和Content- ...

  8. Flash上传超大文件解决方案

    文件夹数据库处理逻辑 public class DbFolder { JSONObject root; public DbFolder() { this.root = new JSONObject() ...

  9. asp.net上传超大文件解决方案

    ASP.NET上传文件用FileUpLoad就可以,但是对文件夹的操作却不能用FileUpLoad来实现. 下面这个示例便是使用ASP.NET来实现上传文件夹并对文件夹进行压缩以及解压. ASP.NE ...

随机推荐

  1. Yii源码分享-底层+view层1

    文件:https://files.cnblogs.com/files/cwlife/YII%E7%BB%A7%E6%89%BF%E6%A0%91.xmind.zip 视屏:https://v.qq.c ...

  2. django 中 null=True 和 blank=True的区别!

    null 是针对数据库而言,如果 null=True, 表示数据库的该字段可以为空,即在Null字段显示为YES. blank 是针对表单的,如果 blank=True,表示你的前端表单填写该字段的时 ...

  3. C#面向对象10 继承

    1.继承: **** 我们可能会在一些类中,写一些重复的成员.我们可以将这些重复的成员,单独的封装到一个类中,作为这些类的父类. Student,Teacher,Driver ----子类  派生类 ...

  4. 如果你的评论被WordPress的Akismet插件屏蔽,怎么解封?

    Akismet是Matt Mullenweg早期创办的一个项目,现在已经是Automattic公司的一个专注于剿杀垃圾评论的产品.在Wordpress用户中使用最多,z-blog也有用户在用,由于垃圾 ...

  5. HTML之盒子变形动画

    4个圆形球作圆周运动 代码: <div class="box"> <div class="box1"></div> < ...

  6. 微信小程序常用事件

    bind bind事件绑定不会阻止冒泡事件向上冒泡,catch事件绑定可以阻止冒泡事件向上冒泡. bindtap  跳转页面 bindchange  .value 改变时触发 change 事件 bi ...

  7. mysql 触发器 if then elseif else 的运用

    create procedure dbname.proc_getGrade (stu_no varchar(20),cour_no varchar(10)) BEGIN declare stu_gra ...

  8. 【Java】 BIO与NIO以及AIO分析

    一.BIO与NIO以及AIO的概念 BIO是同步阻塞式的IO NIO是同步非阻塞的IO (NIO1.0,JDK1.4) AIO是非同步非阻塞的IO(NIO2.0,JDK1.7) 二.BIO简单分析 1 ...

  9. 第十五章、线程之queue模块的各种队列

    目录 第十五章.线程之queue模块的各种队列 一.Queue 二.LifoQueue堆栈 三.PriorityQueue优先级队列 第十五章.线程之queue模块的各种队列 一.Queue impo ...

  10. gyp ERR! stack Error: EACCES: permission denied, mkdir问题解决方案

    sudo npm i --unsafe-perm 原因还是权限问题 就是说 npm 出于安全考虑不支持以 root 用户运行,即使你用 root 用户身份运行了,npm 会自动转成一个叫 nobody ...