1,项目调研

因为需要研究下断点上传的问题。找了很久终于找到一个比较好的项目。

在GoogleCode上面,代码弄下来超级不方便,还是配置hosts才好,把代码重新上传到了github上面。

https://github.com/freewebsys/java-large-file-uploader-demo

效果:

上传中,显示进度,时间,百分比。

点击【Pause】暂停,点击【Resume】继续。

2,代码分析

原始项目:

https://code.google.com/p/java-large-file-uploader/

这个项目最后更新的时间是 2012 年,项目进行了封装使用最简单的方法实现了http的断点上传。

因为html5 里面有读取文件分割文件的类库,所以才可以支持断点上传,所以这个只能在html5 支持的浏览器上面展示。

同时,在js 和 java 同时使用 cr32 进行文件块的校验,保证数据上传正确。

代码在使用了最新的servlet 3.0 的api,使用了异步执行,监听等方法。

上传类UploadServlet

@Component("javaLargeFileUploaderServlet")

@WebServlet(name = "javaLargeFileUploaderServlet", urlPatterns = { "/javaLargeFileUploaderServlet" })

public class UploadServlet extends HttpRequestHandlerServlet

implements HttpRequestHandler {

private static final Logger log = LoggerFactory.getLogger(UploadServlet.class);

@Autowired

UploadProcessor uploadProcessor;

@Autowired

FileUploaderHelper fileUploaderHelper;

@Autowired

ExceptionCodeMappingHelper exceptionCodeMappingHelper;

@Autowired

Authorizer authorizer;

@Autowired

StaticStateIdentifierManager staticStateIdentifierManager;

@Override

public void handleRequest(HttpServletRequest request, HttpServletResponse response)

throws IOException {

log.trace("Handling request");

Serializable jsonObject = null;

try {

// extract the action from the request

UploadServletAction actionByParameterName =

UploadServletAction.valueOf(fileUploaderHelper.getParameterValue(request, UploadServletParameter.action));

// check authorization

checkAuthorization(request, actionByParameterName);

// then process the asked action

jsonObject = processAction(actionByParameterName, request);

// if something has to be written to the response

if (jsonObject != null) {

fileUploaderHelper.writeToResponse(jsonObject, response);

}

}

// If exception, write it

catch (Exception e) {

exceptionCodeMappingHelper.processException(e, response);

}

}

private void checkAuthorization(HttpServletRequest request, UploadServletAction actionByParameterName)

throws MissingParameterException, AuthorizationException {

// check authorization

// if its not get progress (because we do not really care about authorization for get

// progress and it uses an array of file ids)

if (!actionByParameterName.equals(UploadServletAction.getProgress)) {

// extract uuid

final String fileIdFieldValue = fileUploaderHelper.getParameterValue(request, UploadServletParameter.fileId, false);

// if this is init, the identifier is the one in parameter

UUID clientOrJobId;

String parameter = fileUploaderHelper.getParameterValue(request, UploadServletParameter.clientId, false);

if (actionByParameterName.equals(UploadServletAction.getConfig) && parameter != null) {

clientOrJobId = UUID.fromString(parameter);

}

// if not, get it from manager

else {

clientOrJobId = staticStateIdentifierManager.getIdentifier();

}

// call authorizer

authorizer.getAuthorization(

request,

actionByParameterName,

clientOrJobId,

fileIdFieldValue != null ? getFileIdsFromString(fileIdFieldValue).toArray(new UUID[] {}) : null);

}

}

private Serializable processAction(UploadServletAction actionByParameterName, HttpServletRequest request)

throws Exception {

log.debug("Processing action " + actionByParameterName.name());

Serializable returnObject = null;

switch (actionByParameterName) {

case getConfig:

String parameterValue = fileUploaderHelper.getParameterValue(request, UploadServletParameter.clientId, false);

returnObject =

uploadProcessor.getConfig(

parameterValue != null ? UUID.fromString(parameterValue) : null);

break;

case verifyCrcOfUncheckedPart:

returnObject = verifyCrcOfUncheckedPart(request);

break;

case prepareUpload:

returnObject = prepareUpload(request);

break;

case clearFile:

uploadProcessor.clearFile(UUID.fromString(fileUploaderHelper.getParameterValue(request, UploadServletParameter.fileId)));

break;

case clearAll:

uploadProcessor.clearAll();

break;

case pauseFile:

List<UUID> uuids = getFileIdsFromString(fileUploaderHelper.getParameterValue(request, UploadServletParameter.fileId));

uploadProcessor.pauseFile(uuids);

break;

case resumeFile:

returnObject =

uploadProcessor.resumeFile(UUID.fromString(fileUploaderHelper.getParameterValue(request, UploadServletParameter.fileId)));

break;

case setRate:

uploadProcessor.setUploadRate(UUID.fromString(fileUploaderHelper.getParameterValue(request, UploadServletParameter.fileId)),

Long.valueOf(fileUploaderHelper.getParameterValue(request, UploadServletParameter.rate)));

break;

case getProgress:

returnObject = getProgress(request);

break;

}

return returnObject;

}

List<UUID> getFileIdsFromString(String fileIds) {

String[] splittedFileIds = fileIds.split(",");

List<UUID> uuids = Lists.newArrayList();

for (int i = 0; i < splittedFileIds.length; i++) {

uuids.add(UUID.fromString(splittedFileIds[i]));

}

return uuids;

}

private Serializable getProgress(HttpServletRequest request)

throws MissingParameterException {

Serializable returnObject;

String[] ids =

new Gson()

.fromJson(fileUploaderHelper.getParameterValue(request, UploadServletParameter.fileId), String[].class);

Collection<UUID> uuids = Collections2.transform(Arrays.asList(ids), new Function<String, UUID>() {

@Override

public UUID apply(String input) {

return UUID.fromString(input);

}

});

returnObject = Maps.newHashMap();

for (UUID fileId : uuids) {

try {

ProgressJson progress = uploadProcessor.getProgress(fileId);

((HashMap<String, ProgressJson>) returnObject).put(fileId.toString(), progress);

}

catch (FileNotFoundException e) {

log.debug("No progress will be retrieved for " + fileId + " because " + e.getMessage());

}

}

return returnObject;

}

private Serializable prepareUpload(HttpServletRequest request)

throws MissingParameterException, IOException {

// extract file information

PrepareUploadJson[] fromJson =

new Gson()

.fromJson(fileUploaderHelper.getParameterValue(request, UploadServletParameter.newFiles), PrepareUploadJson[].class);

// prepare them

final HashMap<String, UUID> prepareUpload = uploadProcessor.prepareUpload(fromJson);

// return them

return Maps.newHashMap(Maps.transformValues(prepareUpload, new Function<UUID, String>() {

public String apply(UUID input) {

return input.toString();

};

}));

}

private Boolean verifyCrcOfUncheckedPart(HttpServletRequest request)

throws IOException, MissingParameterException, FileCorruptedException, FileStillProcessingException {

UUID fileId = UUID.fromString(fileUploaderHelper.getParameterValue(request, UploadServletParameter.fileId));

try {

uploadProcessor.verifyCrcOfUncheckedPart(fileId,

fileUploaderHelper.getParameterValue(request, UploadServletParameter.crc));

}

catch (InvalidCrcException e) {

// no need to log this exception, a fallback behaviour is defined in the

// throwing method.

// but we need to return something!

return Boolean.FALSE;

}

return Boolean.TRUE;

}

}

异步上传UploadServletAsync

@Component("javaLargeFileUploaderAsyncServlet")

@WebServlet(name = "javaLargeFileUploaderAsyncServlet", urlPatterns = { "/javaLargeFileUploaderAsyncServlet" }, asyncSupported = true)

public class UploadServletAsync extends HttpRequestHandlerServlet

implements HttpRequestHandler {

private static final Logger log = LoggerFactory.getLogger(UploadServletAsync.class);

@Autowired

ExceptionCodeMappingHelper exceptionCodeMappingHelper;

@Autowired

UploadServletAsyncProcessor uploadServletAsyncProcessor;

@Autowired

StaticStateIdentifierManager staticStateIdentifierManager;

@Autowired

StaticStateManager<StaticStatePersistedOnFileSystemEntity> staticStateManager;

@Autowired

FileUploaderHelper fileUploaderHelper;

@Autowired

Authorizer authorizer;

/**

* Maximum time that a streaming request can take.<br>

*/

private long taskTimeOut = DateUtils.MILLIS_PER_HOUR;

@Override

public void handleRequest(final HttpServletRequest request, final HttpServletResponse response)

throws ServletException, IOException {

// process the request

try {

//check if uploads are allowed

if (!uploadServletAsyncProcessor.isEnabled()) {

throw new UploadIsCurrentlyDisabled();

}

// extract stuff from request

final FileUploadConfiguration process = fileUploaderHelper.extractFileUploadConfiguration(request);

log.debug("received upload request with config: "+process);

// verify authorization

final UUID clientId = staticStateIdentifierManager.getIdentifier();

authorizer.getAuthorization(request, UploadServletAction.upload, clientId, process.getFileId());

//check if that file is not paused

if (uploadServletAsyncProcessor.isFilePaused(process.getFileId())) {

log.debug("file "+process.getFileId()+" is paused, ignoring async request.");

return;

}

// get the model

StaticFileState fileState = staticStateManager.getEntityIfPresent().getFileStates().get(process.getFileId());

if (fileState == null) {

throw new FileNotFoundException("File with id " + process.getFileId() + " not found");

}

// process the request asynchronously

final AsyncContext asyncContext = request.startAsync();

asyncContext.setTimeout(taskTimeOut);

// add a listener to clear bucket and close inputstream when process is complete or

// with

// error

asyncContext.addListener(new UploadServletAsyncListenerAdapter(process.getFileId()) {

@Override

void clean() {

log.debug("request " + request + " completed.");

// we do not need to clear the inputstream here.

// and tell processor to clean its shit!

uploadServletAsyncProcessor.clean(clientId, process.getFileId());

}

});

// then process

uploadServletAsyncProcessor.process(fileState, process.getFileId(), process.getCrc(), process.getInputStream(),

new WriteChunkCompletionListener() {

@Override

public void success() {

asyncContext.complete();

}

@Override

public void error(Exception exception) {

// handles a stream ended unexpectedly , it just means the user has

// stopped the

// stream

if (exception.getMessage() != null) {

if (exception.getMessage().equals("Stream ended unexpectedly")) {

log.warn("User has stopped streaming for file " + process.getFileId());

}

else if (exception.getMessage().equals("User cancellation")) {

log.warn("User has cancelled streaming for file id " + process.getFileId());

// do nothing

}

else {

exceptionCodeMappingHelper.processException(exception, response);

}

}

else {

exceptionCodeMappingHelper.processException(exception, response);

}

asyncContext.complete();

}

});

}

catch (Exception e) {

exceptionCodeMappingHelper.processException(e, response);

}

}

}

3,请求流程图:

主要思路就是将文件切分,然后分块上传。

详细代码可以参照我写的这篇文章:

http://blog.ncmem.com/wordpress/2019/08/12/java-http%E5%A4%A7%E6%96%87%E4%BB%B6%E6%96%AD%E7%82%B9%E7%BB%AD%E4%BC%A0%E4%B8%8A%E4%BC%A0/

Java大文件上传详解及实例代码的更多相关文章

  1. java 大文件上传 断点续传 完整版实例 (Socket、IO流)

    ava两台服务器之间,大文件上传(续传),采用了Socket通信机制以及JavaIO流两个技术点,具体思路如下: 实现思路: 1.服:利用ServerSocket搭建服务器,开启相应端口,进行长连接操 ...

  2. Multipart/form-data POST文件上传详解

    Multipart/form-data POST文件上传详解 理论 简单的HTTP POST 大家通过HTTP向服务器发送POST请求提交数据,都是通过form表单提交的,代码如下: <form ...

  3. Multipart/form-data POST文件上传详解(转)

    Multipart/form-data POST文件上传详解 理论 简单的HTTP POST 大家通过HTTP向服务器发送POST请求提交数据,都是通过form表单提交的,代码如下: <form ...

  4. java大文件上传解决方案

    最近遇见一个需要上传百兆大文件的需求,调研了七牛和腾讯云的切片分段上传功能,因此在此整理前端大文件上传相关功能的实现. 在某些业务中,大文件上传是一个比较重要的交互场景,如上传入库比较大的Excel表 ...

  5. java+大文件上传解决方案

    众所皆知,web上传大文件,一直是一个痛.上传文件大小限制,页面响应时间超时.这些都是web开发所必须直面的. 本文给出的解决方案是:前端实现数据流分片长传,后面接收完毕后合并文件的思路. 实现文件夹 ...

  6. 【转】JSch - Java实现的SFTP(文件上传详解篇)

    JSch是Java Secure Channel的缩写.JSch是一个SSH2的纯Java实现.它允许你连接到一个SSH服务器,并且可以使用端口转发,X11转发,文件传输等,当然你也可以集成它的功能到 ...

  7. JSch - Java实现的SFTP(文件上传详解篇)

    JSch是Java Secure Channel的缩写.JSch是一个SSH2的纯Java实现.它允许你连接到一个SSH服务器,并且可以使用端口转发,X11转发,文件传输等,当然你也可以集成它的功能到 ...

  8. JSch - Java实现的SFTP(文件上传详解篇) [转载]

    文章来源:http://www.cnblogs.com/longyg/archive/2012/06/25/2556576.html JSch是Java Secure Channel的缩写.JSch是 ...

  9. JSch - Java实现的SFTP(文件上传详解篇)(转)

    JSch是Java Secure Channel的缩写.JSch是一个SSH2的纯Java实现.它允许你连接到一个SSH服务器,并且可以使用端口转发,X11转发,文件传输等,当然你也可以集成它的功能到 ...

随机推荐

  1. 第六周课程总结&实验报告

    一.实验目的 (1)掌握类的继承 (2)变量的继承和覆盖,方法的继承,重载和覆盖的实现: 二.实验的内容 (1)根据下面的要求实现圆类Circle. 1.圆类Circle的成员变量:radius表示圆 ...

  2. [Git] 020 stash —— Git 中的”皮姆粒子“

    0. 我准备模拟两种情况 第一种 第二种 情况简析 仓库中的最新版本发现 Bug,需要立即修复 当前在 "dev" 分支中工作到一定程度,尚不能提交,但删之可惜 ps: 在分支中没 ...

  3. The Frog's Games

    The Frog's Games Problem Description The annual Games in frogs' kingdom started again. The most famo ...

  4. P1162填涂颜色

    这还是一个搜索题,难度较低,但我提交第三次才AC.. 观察0地图左上角的上面和左面都是一,所以先把他找粗来,然后设成start,然后dfs找到与他联通的块,涂成2即可.再说一下自己犯的低级错误:1.当 ...

  5. 实例学习——爬取豆瓣网TOP250数据

    开发环境:(Windows)eclipse+pydev 网址:https://book.douban.com/top250?start=0 from lxml import etree #解析提取数据 ...

  6. 请写出一段Python代码实现删除一个list里面的重复元素?

    方法1:使用set函数  s=set(list),然后再list(s) 方法2:append    def delList(L): L1 = [] for i in L: if i not in L1 ...

  7. nginx运行出现 file not found 错误处理原因

    在阿里云装nginx+php+mysql nginx运行出现 file not found 错误处理原因 1,第一情况 location ~ \.php$ { # root html; fastcgi ...

  8. python查询mysql中是否存在某张表(传参)

    客户端输入了表的名字,服务端判断表是否存在. 参考:https://blog.csdn.net/qq_36523839/article/details/80639297 需要导入re模块: impor ...

  9. thinkphp5发送邮件(实例代码 非常适合新手)

    第一步:在(https://pan.baidu.com/s/1Fq6lONHlft5D6jvOnNwtoA)下载 phpmailer.rar 解压 然后把文件放入 vendor目录下 第二步:在 ap ...

  10. lldb调试命令

    XCode4.0以后,编译器换成了LLVM 编译器 2.0 与以前相比,更加强大:1.LLVM 编译器是下一带开源的编译技术.完全支持C, Objective-C, 和 C++.2.LLVM 速度比 ...