因为需要研究下断点上传的问题。找了很久终于找到一个比较好的项目。

效果：

上传中，显示进度，时间，百分比。

点击【Pause】暂停，点击【Resume】继续。

2，代码分析

项目进行了封装使用最简单的方法实现了http的断点上传。

因为html5 里面有读取文件分割文件的类库，所以才可以支持断点上传，所以这个只能在html5 支持的浏览器上面展示。

同时，在js 和 java 同时使用 cr32 进行文件块的校验，保证数据上传正确。

代码在使用了最新的servlet 3.0 的api，使用了异步执行，监听等方法。

上传类UploadServlet

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
@Component("javaLargeFileUploaderServlet") 
@WebServlet(name = "javaLargeFileUploaderServlet", urlPatterns = { "/javaLargeFileUploaderServlet" }) 
public class UploadServlet extends HttpRequestHandlerServlet 
        implements HttpRequestHandler { 
   
    private static final Logger log = LoggerFactory.getLogger(UploadServlet.class); 
   
    @Autowired 
    UploadProcessor uploadProcessor; 
   
    @Autowired 
    FileUploaderHelper fileUploaderHelper; 
   
    @Autowired 
    ExceptionCodeMappingHelper exceptionCodeMappingHelper; 
   
    @Autowired 
    Authorizer authorizer; 
   
    @Autowired 
    StaticStateIdentifierManager staticStateIdentifierManager; 
   
   
   
    @Override 
    public void handleRequest(HttpServletRequest request, HttpServletResponse response) 
            throws IOException { 
        log.trace("Handling request"); 
   
        Serializable jsonObject = null; 
        try { 
            // extract the action from the request 
            UploadServletAction actionByParameterName = 
                    UploadServletAction.valueOf(fileUploaderHelper.getParameterValue(request, UploadServletParameter.action)); 
   
            // check authorization 
            checkAuthorization(request, actionByParameterName); 
   
            // then process the asked action 
            jsonObject = processAction(actionByParameterName, request); 
   
   
            // if something has to be written to the response 
            if (jsonObject != null) { 
                fileUploaderHelper.writeToResponse(jsonObject, response); 
            } 
   
        } 
        // If exception, write it 
        catch (Exception e) { 
            exceptionCodeMappingHelper.processException(e, response); 
        } 
   
    } 
   
   
    private void checkAuthorization(HttpServletRequest request, UploadServletAction actionByParameterName) 
            throws MissingParameterException, AuthorizationException { 
   
        // check authorization 
        // if its not get progress (because we do not really care about authorization for get 
        // progress and it uses an array of file ids) 
        if (!actionByParameterName.equals(UploadServletAction.getProgress)) { 
   
            // extract uuid 
            final String fileIdFieldValue = fileUploaderHelper.getParameterValue(request, UploadServletParameter.fileId, false); 
   
            // if this is init, the identifier is the one in parameter 
            UUID clientOrJobId; 
            String parameter = fileUploaderHelper.getParameterValue(request, UploadServletParameter.clientId, false); 
            if (actionByParameterName.equals(UploadServletAction.getConfig) && parameter != null) { 
                clientOrJobId = UUID.fromString(parameter); 
            } 
            // if not, get it from manager 
            else { 
                clientOrJobId = staticStateIdentifierManager.getIdentifier(); 
            } 
   
               
            // call authorizer 
            authorizer.getAuthorization( 
                    request, 
                    actionByParameterName, 
                    clientOrJobId, 
                    fileIdFieldValue != null ? getFileIdsFromString(fileIdFieldValue).toArray(new UUID[] {}) : null); 
   
        } 
    } 
   
   
    private Serializable processAction(UploadServletAction actionByParameterName, HttpServletRequest request) 
            throws Exception { 
        log.debug("Processing action " + actionByParameterName.name()); 
   
        Serializable returnObject = null; 
        switch (actionByParameterName) { 
            case getConfig: 
                String parameterValue = fileUploaderHelper.getParameterValue(request, UploadServletParameter.clientId, false); 
                returnObject = 
                        uploadProcessor.getConfig( 
                                parameterValue != null ? UUID.fromString(parameterValue) : null); 
                break; 
            case verifyCrcOfUncheckedPart: 
                returnObject = verifyCrcOfUncheckedPart(request); 
                break; 
            case prepareUpload: 
                returnObject = prepareUpload(request); 
                break; 
            case clearFile: 
                uploadProcessor.clearFile(UUID.fromString(fileUploaderHelper.getParameterValue(request, UploadServletParameter.fileId))); 
                break; 
            case clearAll: 
                uploadProcessor.clearAll(); 
                break; 
            case pauseFile: 
                List<UUID> uuids = getFileIdsFromString(fileUploaderHelper.getParameterValue(request, UploadServletParameter.fileId)); 
                uploadProcessor.pauseFile(uuids); 
                break; 
            case resumeFile: 
                returnObject = 
                        uploadProcessor.resumeFile(UUID.fromString(fileUploaderHelper.getParameterValue(request, UploadServletParameter.fileId))); 
                break; 
            case setRate: 
                uploadProcessor.setUploadRate(UUID.fromString(fileUploaderHelper.getParameterValue(request, UploadServletParameter.fileId)), 
                        Long.valueOf(fileUploaderHelper.getParameterValue(request, UploadServletParameter.rate))); 
                break; 
            case getProgress: 
                returnObject = getProgress(request); 
                break; 
        } 
        return returnObject; 
    } 
   
   
    List<UUID> getFileIdsFromString(String fileIds) { 
        String[] splittedFileIds = fileIds.split(","); 
        List<UUID> uuids = Lists.newArrayList(); 
        for (int i = 0; i < splittedFileIds.length; i++) { 
            uuids.add(UUID.fromString(splittedFileIds[i])); 
        }  
        return uuids; 
    } 
   
   
    private Serializable getProgress(HttpServletRequest request) 
            throws MissingParameterException { 
        Serializable returnObject; 
        String[] ids = 
                new Gson() 
                        .fromJson(fileUploaderHelper.getParameterValue(request, UploadServletParameter.fileId), String[].class); 
        Collection<UUID> uuids = Collections2.transform(Arrays.asList(ids), new Function<String, UUID>() { 
   
            @Override 
            public UUID apply(String input) { 
                return UUID.fromString(input); 
            } 
   
        }); 
        returnObject = Maps.newHashMap(); 
        for (UUID fileId : uuids) { 
            try { 
                ProgressJson progress = uploadProcessor.getProgress(fileId); 
                ((HashMap<String, ProgressJson>) returnObject).put(fileId.toString(), progress); 
            } 
            catch (FileNotFoundException e) { 
                log.debug("No progress will be retrieved for " + fileId + " because " + e.getMessage()); 
            } 
        } 
        return returnObject; 
    } 
   
   
    private Serializable prepareUpload(HttpServletRequest request) 
            throws MissingParameterException, IOException { 
   
        // extract file information 
        PrepareUploadJson[] fromJson = 
                new Gson() 
                        .fromJson(fileUploaderHelper.getParameterValue(request, UploadServletParameter.newFiles), PrepareUploadJson[].class); 
   
        // prepare them 
        final HashMap<String, UUID> prepareUpload = uploadProcessor.prepareUpload(fromJson); 
   
        // return them 
        return Maps.newHashMap(Maps.transformValues(prepareUpload, new Function<UUID, String>() { 
   
            public String apply(UUID input) { 
                return input.toString(); 
            }; 
        })); 
    } 
   
   
    private Boolean verifyCrcOfUncheckedPart(HttpServletRequest request) 
            throws IOException, MissingParameterException, FileCorruptedException, FileStillProcessingException { 
        UUID fileId = UUID.fromString(fileUploaderHelper.getParameterValue(request, UploadServletParameter.fileId)); 
        try { 
            uploadProcessor.verifyCrcOfUncheckedPart(fileId, 
                    fileUploaderHelper.getParameterValue(request, UploadServletParameter.crc)); 
        } 
        catch (InvalidCrcException e) { 
            // no need to log this exception, a fallback behaviour is defined in the 
            // throwing method. 
            // but we need to return something! 
            return Boolean.FALSE; 
        } 
        return Boolean.TRUE; 
    } 
}

异步上传UploadServletAsync

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
@Component("javaLargeFileUploaderAsyncServlet") 
@WebServlet(name = "javaLargeFileUploaderAsyncServlet", urlPatterns = { "/javaLargeFileUploaderAsyncServlet" }, asyncSupported = true) 
public class UploadServletAsync extends HttpRequestHandlerServlet 
        implements HttpRequestHandler { 
   
    private static final Logger log = LoggerFactory.getLogger(UploadServletAsync.class); 
   
    @Autowired 
    ExceptionCodeMappingHelper exceptionCodeMappingHelper; 
   
    @Autowired 
    UploadServletAsyncProcessor uploadServletAsyncProcessor; 
       
    @Autowired 
    StaticStateIdentifierManager staticStateIdentifierManager; 
   
    @Autowired 
    StaticStateManager<StaticStatePersistedOnFileSystemEntity> staticStateManager; 
   
    @Autowired 
    FileUploaderHelper fileUploaderHelper; 
   
    @Autowired 
    Authorizer authorizer; 
   
    /**
     * Maximum time that a streaming request can take.<br>
     */ 
    private long taskTimeOut = DateUtils.MILLIS_PER_HOUR; 
   
   
    @Override 
    public void handleRequest(final HttpServletRequest request, final HttpServletResponse response) 
            throws ServletException, IOException { 
   
        // process the request 
        try { 
   
            //check if uploads are allowed 
            if (!uploadServletAsyncProcessor.isEnabled()) { 
                throw new UploadIsCurrentlyDisabled(); 
            } 
               
            // extract stuff from request 
            final FileUploadConfiguration process = fileUploaderHelper.extractFileUploadConfiguration(request); 
   
            log.debug("received upload request with config: "+process); 
   
            // verify authorization 
            final UUID clientId = staticStateIdentifierManager.getIdentifier(); 
            authorizer.getAuthorization(request, UploadServletAction.upload, clientId, process.getFileId()); 
   
            //check if that file is not paused 
            if (uploadServletAsyncProcessor.isFilePaused(process.getFileId())) { 
                log.debug("file "+process.getFileId()+" is paused, ignoring async request."); 
                return; 
            } 
               
            // get the model 
            StaticFileState fileState = staticStateManager.getEntityIfPresent().getFileStates().get(process.getFileId()); 
            if (fileState == null) { 
                throw new FileNotFoundException("File with id " + process.getFileId() + " not found"); 
            } 
   
            // process the request asynchronously 
            final AsyncContext asyncContext = request.startAsync(); 
            asyncContext.setTimeout(taskTimeOut); 
   
   
            // add a listener to clear bucket and close inputstream when process is complete or 
            // with 
            // error 
            asyncContext.addListener(new UploadServletAsyncListenerAdapter(process.getFileId()) { 
   
                @Override 
                void clean() { 
                    log.debug("request " + request + " completed."); 
                    // we do not need to clear the inputstream here. 
                    // and tell processor to clean its shit! 
                    uploadServletAsyncProcessor.clean(clientId, process.getFileId()); 
                } 
            }); 
   
            // then process 
            uploadServletAsyncProcessor.process(fileState, process.getFileId(), process.getCrc(), process.getInputStream(), 
                    new WriteChunkCompletionListener() { 
   
                        @Override 
                        public void success() { 
                            asyncContext.complete(); 
                        } 
   
   
                        @Override 
                        public void error(Exception exception) { 
                            // handles a stream ended unexpectedly , it just means the user has 
                            // stopped the 
                            // stream 
                            if (exception.getMessage() != null) { 
                                if (exception.getMessage().equals("Stream ended unexpectedly")) { 
                                    log.warn("User has stopped streaming for file " + process.getFileId()); 
                                } 
                                else if (exception.getMessage().equals("User cancellation")) { 
                                    log.warn("User has cancelled streaming for file id " + process.getFileId()); 
                                    // do nothing 
                                } 
                                else { 
                                    exceptionCodeMappingHelper.processException(exception, response); 
                                } 
                            } 
                            else { 
                                exceptionCodeMappingHelper.processException(exception, response); 
                            } 
   
                            asyncContext.complete(); 
                        } 
   
                    }); 
        } 
        catch (Exception e) { 
            exceptionCodeMappingHelper.processException(e, response); 
        } 
   
    } 
   
}

3，请求流程图：

主要思路就是将文件切分，然后分块上传。

DEMO下载地址：https://dwz.cn/fgXtRtnu

java http大文件断点续传上传的更多相关文章

  1. java springboot 大文件分片上传处理

    参考自:https://blog.csdn.net/u014150463/article/details/74044467 这里只写后端的代码,基本的思想就是,前端将文件分片,然后每次访问上传接口的时 ...

  2. Ajax+Java实现大文件切割上传

    技术体系:html5(formdata) + java + servlet3.0+maven + tomcat7 <!DOCTYPE html> <html> <head ...

  3. 关于：基于http协议大文件断点续传上传至web服务器

    关键部分 前端用file.slice()分块 前端用FileReader获取每一分块的md5值 后端用MultipartFile接受分块文件 后端用FileOutputStream拼装分块文件 话不多 ...

  4. iOS大文件分片上传和断点续传

    总结一下大文件分片上传和断点续传的问题.因为文件过大(比如1G以上),必须要考虑上传过程网络中断的情况.http的网络请求中本身就已经具备了分片上传功能,当传输的文件比较大时,http协议自动会将文件 ...

  5. js大文件分块上传断点续传demo

    文件夹上传:从前端到后端 文件上传是 Web 开发肯定会碰到的问题,而文件夹上传则更加难缠.网上关于文件夹上传的资料多集中在前端,缺少对于后端的关注,然后讲某个后端框架文件上传的文章又不会涉及文件夹. ...

  6. 使用webuploader组件实现大文件分片上传，断点续传

    本人在2010年时使用swfupload为核心进行文件的批量上传的解决方案.见文章:WEB版一次选择多个文件进行批量上传(swfupload)的解决方案. 本人在2013年时使用plupload为核心 ...

  7. vue大文件分片上传插件

    最近遇见一个需要上传百兆大文件的需求,调研了七牛和腾讯云的切片分段上传功能,因此在此整理前端大文件上传相关功能的实现. 在某些业务中,大文件上传是一个比较重要的交互场景,如上传入库比较大的Excel表 ...

  8. Webuploader 大文件分片上传

    百度Webuploader 大文件分片上传(.net接收)   前阵子要做个大文件上传的功能,找来找去发现Webuploader还不错,关于她的介绍我就不再赘述. 动手前,在园子里找到了一篇不错的分片 ...

  9. Hadoop如何将TB级大文件的上传性能优化上百倍？

    这篇文章,我们来看看,Hadoop的HDFS分布式文件系统的文件上传的性能优化. 首先,我们还是通过一张图来回顾一下文件上传的大概的原理. 由上图所示,文件上传的原理,其实说出来也简单. 比如有个TB ...

随机推荐

  1. gearman中worker常驻后台，导致MySQL server has gone away

    产生这个原因主要有如下几点: 1.mysql服务宕机了 2.长时间没有操作,超过了wait_timeout的设置,mysql自动断开 3.mysql请求链接被主动kill 4.发送的请求或返回结果过大 ...

  2. 我的开发小tip

    开发原则:1.谁开发,谁负责到底.自己的开发的模块自己维护,不要让别人替你维护,否则很麻烦:2.合理分配时间3.谨慎的处理遇到的bug和问题,不是自己开发的不要轻举妄动,提交到待办中即可4.万勿过度设 ...

  3. mysql学习笔记-1.下载与安装

    1.到mysql官网下载操作系统对应的数据库,下载地址https://dev.mysql.com/downloads/mysql/ 2.有msi安装版本和zip压缩版本,2种安装方式不同, 3.安装后 ...

  4. Latex基本用法

    空格 需要使用 \qquad,\quad,\,应该是占位符和变量之间需要有{}相隔. $$ C_{1} \qquad {C_2} $$ $$ C_{1} \quad {C_2} $$ $$ C_{1} ...

  5. MVC中利用knockout.js实现动态uniqueId

    题目比较拗口,但是这篇文章确实直说这一点. knockout.js是一个JS库,它的官网是http://knockoutjs.com/ 这篇文章的重点是knockout在工作的一个功能中的应用.最终效 ...

  6. UVA 11324.The Largest Clique tarjan缩点+拓扑dp

    题目链接:https://vjudge.net/problem/UVA-11324 题意:求一个有向图中结点数最大的结点集,使得该结点集中任意两个结点u和v满足:要目u可以到达v,要么v可以到达u(相 ...

  7. 杭电1518 Square(构成正方形) 搜索

    HDOJ1518 Square Time Limit: 10000/5000 MS (Java/Others)    Memory Limit: 65536/32768 K (Java/Others) ...

  8. ​零基础该如何学习UI设计

    ​零基础学习该如何学习UI设计,没有基础该怎么开始学习呢?UI设计可以说是入行门槛很低的职业了,而且随着互联网的快速发展,UI设计的市场前景也越来也好,更多的人看到了这个高薪的行业也开始心动了,想要在 ...

  9. Unix和Windows文件格式转化

    可能的原因有: 1)执行权限的问题 解决方法: chmod +x ***.py 2)python版本的问题 解决方法:在执行时或者在py文件中选择好对应的Python的版本 3)python文件格式的 ...

  10. 不解压查看tar.gz包内文件

    通过tar命令备份.解压缩文件,也可在不解压缩文件时查看包内的文件信息. 使用如下参数: tar -ztvf file.tar.gz 将列出所有包内的文件列表,包括目录 -z, --gzip, --g ...