场景:一个上传接口,需要上传几十M的文件,文件中包含10几W的数据,然后对10+W的数据进行同步批量插入,每次批量插入1W。最后返回结果。

项目上线一段时间后,上传接口出现问题,数据库用的MySQL5.7.21,报了如下错误:

2018-07-16 01:30:03.497 ERROR com.alibaba.druid.pool.DruidDataSource Line:1594 - discard connection
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure The last packet successfully received from the server was 1,480 milliseconds ago. The last packet sent successfully to the server was 1,480 milliseconds ago.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1121)
at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3603)
at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3492)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4043)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2503)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2664)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2815)
at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2155)
at com.mysql.jdbc.PreparedStatement.execute(PreparedStatement.java:1379)
at com.alibaba.druid.pool.DruidPooledPreparedStatement.execute(DruidPooledPreparedStatement.java:498)
at org.apache.ibatis.executor.statement.PreparedStatementHandler.query(PreparedStatementHandler.java:63)
at org.apache.ibatis.executor.statement.RoutingStatementHandler.query(RoutingStatementHandler.java:79)
at org.apache.ibatis.executor.SimpleExecutor.doQuery(SimpleExecutor.java:63)
at org.apache.ibatis.executor.BaseExecutor.queryFromDatabase(BaseExecutor.java:324)
at org.apache.ibatis.executor.BaseExecutor.query(BaseExecutor.java:156)
at org.apache.ibatis.executor.CachingExecutor.query(CachingExecutor.java:109)
at com.github.pagehelper.PageInterceptor.intercept(PageInterceptor.java:137)
at org.apache.ibatis.plugin.Plugin.invoke(Plugin.java:61)
at com.sun.proxy.$Proxy63.query(Unknown Source)
at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:148)
at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:141)
at sun.reflect.GeneratedMethodAccessor102.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:434)
at com.sun.proxy.$Proxy25.selectList(Unknown Source)
at org.mybatis.spring.SqlSessionTemplate.selectList(SqlSessionTemplate.java:231)
at org.apache.ibatis.binding.MapperMethod.executeForMany(MapperMethod.java:137)
at org.apache.ibatis.binding.MapperMethod.execute(MapperMethod.java:75)
at org.apache.ibatis.binding.MapperProxy.invoke(MapperProxy.java:53)
at com.sun.proxy.$Proxy51.selectReportListByReportDate(Unknown Source)
at com.rxwx.service.report.impl.PrcmtReportServiceImpl.generateProcurementReport(PrcmtReportServiceImpl.java:97)
at com.rxwx.service.report.impl.PrcmtReportServiceImpl$$FastClassBySpringCGLIB$$f325780.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:720)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:280)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:655)
at com.rxwx.service.report.impl.PrcmtReportServiceImpl$$EnhancerBySpringCGLIB$$8c585594.generateProcurementReport(<generated>)
at com.rxwx.service.report.impl.PrcmtReportServiceImpl$$FastClassBySpringCGLIB$$f325780.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:720)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:280)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:655)
at com.rxwx.service.report.impl.PrcmtReportServiceImpl$$EnhancerBySpringCGLIB$$9a830f32.generateProcurementReport(<generated>)
at com.rxwx.task.sup.GenarateReportJob.selfGenReportTask1(GenarateReportJob.java:37)
at com.rxwx.task.sup.GenarateReportJob$$FastClassBySpringCGLIB$$64b31449.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:720)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.aop.interceptor.AsyncExecutionInterceptor$1.call(AsyncExecutionInterceptor.java:115)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.EOFException: Can not read response from server. Expected to read 4 bytes, read 0 bytes before connection was unexpectedly lost.
at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:3052)
at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3503)
... 58 common frames omitted

首先百度了一发,按照网上的解决方案,修改了my.cnf中的超时时间:

wait_timeout=31536000
interactive_timeout=31536000

将原来默认的8小时改为1年。这里单位是秒。

修改了德鲁伊的:testOnBorrow = true。

跑了一天,MMP,凌晨又出现这个错误了。

一个电话把睡梦中的我吵醒,扰我周末扰我梦,没得办法,重启下服务,重启下MySQL,先上传吧,睡觉。

周一到公司谷歌了一发,修改my.cnf的net_read_timeout和net_write_timeout 参数,将这两个参数调大,改为6000。由于项目中的数据存在无效数据,所以删除了10天之前的无效数据,减少了表中的数量。明天看效果。

show global variables like "%timeout%";

  1. connect_timeout

    连接超时 mysql连接共有6次握手,3次TCP协议这个跟connect_timeout参数没有关系,另外3次跟connect_timeout参数有关系,该参数主要是为了防止网络不佳时应用重连导致连接数涨太快,一般默认即可。

  2. delayed_insert_timeout

    这是为MyISAM INSERT DELAY设计的超时参数,在INSERT DELAY中止前等待INSERT语句的时间

  3. interactive_timeout

    服务器关闭交互式连接前等待活动的秒数。交互式客户端定义为在mysql_real_connect()中使用CLIENT_INTERACTIVE选项的客户端。参数默认值:28800秒(8小时)

  4. lock_wait_timeout

    锁等待超时时间

  5. net_read_timeout / net_write_timeout

    这个参数只对TCP/IP链接有效,分别是数据库等待接收客户端发送网络包和发送网络包给客户端的超时时间,这是在Activity状态下的线程才有效的参数

  6. slave_net_timeout

    这是Slave判断主机是否挂掉的超时设置,在设定时间内依然没有获取到Master的回应就人为Master挂掉了

  7. wait_timeout

    服务器关闭非交互连接之前等待活动的秒数。

效果很明显。今天没有出现问题。

The last packet successfully received from the server was 1,480 milliseconds ago.的更多相关文章

  1. The last packet successfully received from the server was 20,519 milliseconds ago. The last packet sent successfully to the server was 0 milliseconds ago.

    本地升级了下MySQL的版本,从5.6升为5.7,数据文件直接拷贝的,项目查询数据库报错: Could not retrieve transation read-only status server ...

  2. 【异常】ser class threw exception: java.sql.SQLException: The last packet successfully received from the server was 39,444 milliseconds ago. The last

    1 详细异常 ser class threw exception: java.sql.SQLException: The last packet successfully received from ...

  3. The last packet successfully received from the server was 39,900 milliseconds ago问题解决

    1,之前用Mysql或者mycat的时候都没有这个问题.后来改为haproxy+keepalived+mycat后出现这个问题 2,网上查了很多说法,我按照网上说的改了 datasource: url ...

  4. The last packet successfully received from the server was 2,926,157 milliseconds ago. The last packet sent successfully to the server was 2,926,158 milliseconds ago. is longer than the server configured value of 'wait_timeout'. 解决办法

    Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: The last packet successfully rec ...

  5. Cause: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: The last packet successfully received from the server was 78,050,512 milliseconds ago.

    今天访问已经架上服务器的网站,报错: Cause: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: The last packet s ...

  6. Communications link failure,The last packet successfully received from the server was *** millisecon

    使用Connector/J连接MySQL数据库,程序运行较长时间后就会报以下错误: Communications link failure,The last packet successfully r ...

  7. Mysql 的异常:The last packet successfully received from the server was 90 milliseconds ago. The last packet sent successfully to the server was 43,603,303 milliseconds ago. is longer than the server con

    调试一个程序, 调试到一半, 下班回家, 程序卡在了某一行, 第二天早上回来一看, 发现了异常: Wed Sep :: GMT+: WARN: Establishing SSL connection ...

  8. 数据库连接超时:“The last packet successfully received from the server was xxx milliseconds ago”

    产生的原因:应用方的数据库连接有效期时间,大于数据库自己设置的有效期. 解决方案: 一.修改druid配置(如果使用druid的话) spring.datasource.druid.validatio ...

  9. The last packet successfully received from the server was 49,061,696 millise

    解决连接:https://blog.csdn.net/pandajava/article/details/41946251

随机推荐

  1. Spring Boot 2实现分布式锁——这才是实现分布式锁的正确姿势!

    参考资料 网址 Spring Boot 2实现分布式锁--这才是实现分布式锁的正确姿势! http://www.spring4all.com/article/6892

  2. cmds在线重定义增加列

    --输出信息采用缩排或换行格式化 EXEC DBMS_METADATA.set_transform_param(DBMS_METADATA.session_transform, 'PRETTY', T ...

  3. 搭建hadoop的HA集群模式(hadoop2.7.3+hive+spark)

    参考:http://blog.51cto.com/12824426/2177663?source=drh 一.集群的规划 Zookeeper集群:192.168.176.131 (bigdata112 ...

  4. CSS进阶之路

    下面主要引用http://www.cnblogs.com/wangfupeng1988/tag/css知多少/ CSS进阶笔记: 一.学习CSS的三个突破点 1.浏览器如何加载和解析CSS——CSS的 ...

  5. Chomp类游戏——必胜策略分析

    首先介绍一个重要定理——策梅洛定理(Zermelo) 策梅洛定理,表明在二人参与的游戏/博弈中,如果满足: --------游戏的步骤数有限 --------信息完备(二人都了解游戏规则,了解游戏曾经 ...

  6. Docker报错“Dockerfile parse error line 1: FROM requires either one or three arguments”

    看官方文档Format: 以 '#' 开头一行被视为评论,出现在其他位置视为参数. 也就不难理解报错原因:将写在同一行的注释视为参数了. 原Dockerfile: 改为:

  7. 华三IRF配置

    配置步骤: IRF的配置步骤: 1.配置IRF域(域ID=10.成员ID.优先级) irf domain 10irf member 1 priority 10irf member 2 priority ...

  8. 使用jenkins exporter 监控jenkins 构建任务

    jenkins 提供了rest api,我们可以基于rest api 暴露prometheus metrics,社区已经有了好多 开源的实现了,使用起来也比较方便,以下集成几个进行简单的测试 环境准备 ...

  9. 使用docker部署微服务

    https://my.oschina.net/silenceyawen/blog/1819472 http://jvm123.com/2019/08/docker-shi-yong.html 从201 ...

  10. 2018传智黑马Python人工智能视频教程(基础+就业+面试)

    2018传智黑马Python人工智能视频教程(基础+就业+面试) 2018传智黑马Python人工智能视频教程(基础+就业+面试) 2018传智黑马Python人工智能视频教程(基础+就业+面试) 下 ...