场景:一个上传接口,需要上传几十M的文件,文件中包含10几W的数据,然后对10+W的数据进行同步批量插入,每次批量插入1W。最后返回结果。

项目上线一段时间后,上传接口出现问题,数据库用的MySQL5.7.21,报了如下错误:

2018-07-16 01:30:03.497 ERROR com.alibaba.druid.pool.DruidDataSource Line:1594 - discard connection
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure The last packet successfully received from the server was 1,480 milliseconds ago. The last packet sent successfully to the server was 1,480 milliseconds ago.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1121)
at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3603)
at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3492)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4043)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2503)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2664)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2815)
at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2155)
at com.mysql.jdbc.PreparedStatement.execute(PreparedStatement.java:1379)
at com.alibaba.druid.pool.DruidPooledPreparedStatement.execute(DruidPooledPreparedStatement.java:498)
at org.apache.ibatis.executor.statement.PreparedStatementHandler.query(PreparedStatementHandler.java:63)
at org.apache.ibatis.executor.statement.RoutingStatementHandler.query(RoutingStatementHandler.java:79)
at org.apache.ibatis.executor.SimpleExecutor.doQuery(SimpleExecutor.java:63)
at org.apache.ibatis.executor.BaseExecutor.queryFromDatabase(BaseExecutor.java:324)
at org.apache.ibatis.executor.BaseExecutor.query(BaseExecutor.java:156)
at org.apache.ibatis.executor.CachingExecutor.query(CachingExecutor.java:109)
at com.github.pagehelper.PageInterceptor.intercept(PageInterceptor.java:137)
at org.apache.ibatis.plugin.Plugin.invoke(Plugin.java:61)
at com.sun.proxy.$Proxy63.query(Unknown Source)
at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:148)
at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:141)
at sun.reflect.GeneratedMethodAccessor102.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:434)
at com.sun.proxy.$Proxy25.selectList(Unknown Source)
at org.mybatis.spring.SqlSessionTemplate.selectList(SqlSessionTemplate.java:231)
at org.apache.ibatis.binding.MapperMethod.executeForMany(MapperMethod.java:137)
at org.apache.ibatis.binding.MapperMethod.execute(MapperMethod.java:75)
at org.apache.ibatis.binding.MapperProxy.invoke(MapperProxy.java:53)
at com.sun.proxy.$Proxy51.selectReportListByReportDate(Unknown Source)
at com.rxwx.service.report.impl.PrcmtReportServiceImpl.generateProcurementReport(PrcmtReportServiceImpl.java:97)
at com.rxwx.service.report.impl.PrcmtReportServiceImpl$$FastClassBySpringCGLIB$$f325780.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:720)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:280)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:655)
at com.rxwx.service.report.impl.PrcmtReportServiceImpl$$EnhancerBySpringCGLIB$$8c585594.generateProcurementReport(<generated>)
at com.rxwx.service.report.impl.PrcmtReportServiceImpl$$FastClassBySpringCGLIB$$f325780.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:720)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:280)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:655)
at com.rxwx.service.report.impl.PrcmtReportServiceImpl$$EnhancerBySpringCGLIB$$9a830f32.generateProcurementReport(<generated>)
at com.rxwx.task.sup.GenarateReportJob.selfGenReportTask1(GenarateReportJob.java:37)
at com.rxwx.task.sup.GenarateReportJob$$FastClassBySpringCGLIB$$64b31449.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:720)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.aop.interceptor.AsyncExecutionInterceptor$1.call(AsyncExecutionInterceptor.java:115)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.EOFException: Can not read response from server. Expected to read 4 bytes, read 0 bytes before connection was unexpectedly lost.
at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:3052)
at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3503)
... 58 common frames omitted

首先百度了一发,按照网上的解决方案,修改了my.cnf中的超时时间:

wait_timeout=31536000
interactive_timeout=31536000

将原来默认的8小时改为1年。这里单位是秒。

修改了德鲁伊的:testOnBorrow = true。

跑了一天,MMP,凌晨又出现这个错误了。

一个电话把睡梦中的我吵醒,扰我周末扰我梦,没得办法,重启下服务,重启下MySQL,先上传吧,睡觉。

周一到公司谷歌了一发,修改my.cnf的net_read_timeout和net_write_timeout 参数,将这两个参数调大,改为6000。由于项目中的数据存在无效数据,所以删除了10天之前的无效数据,减少了表中的数量。明天看效果。

show global variables like "%timeout%";

  1. connect_timeout

    连接超时 mysql连接共有6次握手,3次TCP协议这个跟connect_timeout参数没有关系,另外3次跟connect_timeout参数有关系,该参数主要是为了防止网络不佳时应用重连导致连接数涨太快,一般默认即可。

  2. delayed_insert_timeout

    这是为MyISAM INSERT DELAY设计的超时参数,在INSERT DELAY中止前等待INSERT语句的时间

  3. interactive_timeout

    服务器关闭交互式连接前等待活动的秒数。交互式客户端定义为在mysql_real_connect()中使用CLIENT_INTERACTIVE选项的客户端。参数默认值:28800秒(8小时)

  4. lock_wait_timeout

    锁等待超时时间

  5. net_read_timeout / net_write_timeout

    这个参数只对TCP/IP链接有效,分别是数据库等待接收客户端发送网络包和发送网络包给客户端的超时时间,这是在Activity状态下的线程才有效的参数

  6. slave_net_timeout

    这是Slave判断主机是否挂掉的超时设置,在设定时间内依然没有获取到Master的回应就人为Master挂掉了

  7. wait_timeout

    服务器关闭非交互连接之前等待活动的秒数。

效果很明显。今天没有出现问题。

The last packet successfully received from the server was 1,480 milliseconds ago.的更多相关文章

  1. The last packet successfully received from the server was 20,519 milliseconds ago. The last packet sent successfully to the server was 0 milliseconds ago.

    本地升级了下MySQL的版本,从5.6升为5.7,数据文件直接拷贝的,项目查询数据库报错: Could not retrieve transation read-only status server ...

  2. 【异常】ser class threw exception: java.sql.SQLException: The last packet successfully received from the server was 39,444 milliseconds ago. The last

    1 详细异常 ser class threw exception: java.sql.SQLException: The last packet successfully received from ...

  3. The last packet successfully received from the server was 39,900 milliseconds ago问题解决

    1,之前用Mysql或者mycat的时候都没有这个问题.后来改为haproxy+keepalived+mycat后出现这个问题 2,网上查了很多说法,我按照网上说的改了 datasource: url ...

  4. The last packet successfully received from the server was 2,926,157 milliseconds ago. The last packet sent successfully to the server was 2,926,158 milliseconds ago. is longer than the server configured value of 'wait_timeout'. 解决办法

    Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: The last packet successfully rec ...

  5. Cause: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: The last packet successfully received from the server was 78,050,512 milliseconds ago.

    今天访问已经架上服务器的网站,报错: Cause: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: The last packet s ...

  6. Communications link failure,The last packet successfully received from the server was *** millisecon

    使用Connector/J连接MySQL数据库,程序运行较长时间后就会报以下错误: Communications link failure,The last packet successfully r ...

  7. Mysql 的异常:The last packet successfully received from the server was 90 milliseconds ago. The last packet sent successfully to the server was 43,603,303 milliseconds ago. is longer than the server con

    调试一个程序, 调试到一半, 下班回家, 程序卡在了某一行, 第二天早上回来一看, 发现了异常: Wed Sep :: GMT+: WARN: Establishing SSL connection ...

  8. 数据库连接超时:“The last packet successfully received from the server was xxx milliseconds ago”

    产生的原因:应用方的数据库连接有效期时间,大于数据库自己设置的有效期. 解决方案: 一.修改druid配置(如果使用druid的话) spring.datasource.druid.validatio ...

  9. The last packet successfully received from the server was 49,061,696 millise

    解决连接:https://blog.csdn.net/pandajava/article/details/41946251

随机推荐

  1. c#中的解析HTML组件 -- (HtmlAgilityPack,Jumony,ScrapySharp,NSoup,Fizzler)

    做数据抓取,网络爬虫方面的开发,自然少不了解析HTML源码的操作.那么问题来了,到底.NET如何来解析HTML,有哪些解析HTML源码的好用的,有效的组件呢?   作者在开始做这方面开发的时候就被这些 ...

  2. Spring Boot 缓存 知识点

    每次调用需要缓存功能的方法时,Spring会检查指定参数的指定的目标方法是否已经被调用过:如果有就直接从缓存中获取方法调用后的结果,如果没有就调用方法并缓存结果后返回给用户.下次调用直接从缓存中获取. ...

  3. Flutter 数据存储之 shared_preferences

    资源名称 网址 github https://github.com/flutter/plugins/tree/master/packages/shared_preferences Flutter 数据 ...

  4. 将表格转化为Latex代码的在线工具

    这个在线工具的网址为:http://www.tablesgenerator.com/latex_tables,好用.

  5. LVS是什么及作用?

    LVS的概念和重要性 LVS: 概念:是Linux Virtual Server的简写,意即Linux虚拟服务器,是一个虚拟的服务器集群系统 作用:举例 像有三个小区,但是工作的时间和休息的时间不一样 ...

  6. Linux 系统管理——Linux文件系统与日志

    1.inode 包含文件的元信息(1)inode 内容:文件的字节数.拥有者的 UID.GID.文件的读写执行权限.时间戳等,但不包含文件名.文件名是储存在目录的目录项中.(2)查看文件的 inode ...

  7. ACM数据结构-线段树

    1.维护区间最大最小值模板(以维护最小值为例) #include<iostream> #include<stdio.h> #define LEN 11 #define MAX ...

  8. A*G/C011

    A*G/C011 A Airport Bus 不会zbl/kk B Colorful Creatures 枚举每个开始的点直接倍增 我好像sb了,可行的是一段前缀所以可以直接2分 C Squared ...

  9. 利用 Matplotlib 绘图

    各类绘图 ## 导入包 import matplotlib as mpl import matplotlib.pyplot as plt import seaborn as sns ## 参数设置 # ...

  10. JS简单获取当前日期时间的方法(yyyy-MM-dd hh:mm:ss)

    <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xht ...