一、异常出现的场景

  一次线上订单历史数据字段刷新操作,3张表100多万数据。由于同步更新太慢大概20分钟以上,所以采用异不的方式。代码如下:

  1. private void batchUpdate(List<SaasOrderRecordDataForUpdate> saasOrderRecordDataForUpdateList, List<SaasServiceOrderInfoDataForUpdate> saasServiceOrderInfoDataForUpdateList, List<OrderGoodsDataForUpdate> orderGoodsDataForUpdateList, List<OrderAdditionCostInfoDataForUpdate> orderAdditionCostInfoDataForUpdateList) {
  2. List<Future> asyncResultList = new ArrayList<>();
  3. if (CollectionUtils.isNotEmpty(saasOrderRecordDataForUpdateList)) {
  4. int size = saasOrderRecordDataForUpdateList.size();
  5. int count = size / ;
  6. for (int i = ; i <= count; i++) {
  7. List<SaasOrderRecordDataForUpdate> subList = null;
  8. if (i == count) {
  9. if ( * i < size) {
  10. subList = saasOrderRecordDataForUpdateList.subList( * i, size);
  11. }
  12. } else {
  13. subList = saasOrderRecordDataForUpdateList.subList( * i, * (i + ));
  14. }
  15.  
  16. if (CollectionUtils.isNotEmpty(subList)) {
  17. Future future = dataRefreshJobServiceForAsync.batchUpdateSaasOrderRecordDataForUpdate(subList);
  18. asyncResultList.add(future);
  19. }
  20. }
  21. }
  22. XxlJobLogger.log("批量更新订单数:{}", saasOrderRecordDataForUpdateList.size());
  23. if (CollectionUtils.isNotEmpty(saasServiceOrderInfoDataForUpdateList)) {
  24. int size = saasServiceOrderInfoDataForUpdateList.size();
  25. int count = size / ;
  26. for (int i = ; i <= count; i++) {
  27. List<SaasServiceOrderInfoDataForUpdate> subList = null;
  28. if (i == count) {
  29. if ( * i < size) {
  30. subList = saasServiceOrderInfoDataForUpdateList.subList( * i, size);
  31. }
  32. } else {
  33. subList = saasServiceOrderInfoDataForUpdateList.subList( * i, * (i + ));
  34. }
  35.  
  36. if (CollectionUtils.isNotEmpty(subList)) {
  37. Future future = dataRefreshJobServiceForAsync.batchUpdateSaasServiceOrderInfoDataForUpdate(subList);
  38. asyncResultList.add(future);
  39. }
  40. }
  41.  
  42. }
  43. XxlJobLogger.log("批量更新订单服务数:{}", saasServiceOrderInfoDataForUpdateList.size());
  44. if (CollectionUtils.isNotEmpty(orderGoodsDataForUpdateList)) {
  45. int size = orderGoodsDataForUpdateList.size();
  46. int count = size / ;
  47. for (int i = ; i <= count; i++) {
  48. List<OrderGoodsDataForUpdate> subList = null;
  49. if (i == count) {
  50. if ( * i < size) {
  51. subList = orderGoodsDataForUpdateList.subList( * i, size);
  52. }
  53. } else {
  54. subList = orderGoodsDataForUpdateList.subList( * i, * (i + ));
  55. }
  56.  
  57. if (CollectionUtils.isNotEmpty(subList)) {
  58. Future future = dataRefreshJobServiceForAsync.batchUpdateOrderGoodsDataForUpdate(subList);
  59. asyncResultList.add(future);
  60. }
  61. }
  62.  
  63. }
  64. XxlJobLogger.log("批量更新订单商品数:{}", orderGoodsDataForUpdateList.size());
  65. if (CollectionUtils.isNotEmpty(orderAdditionCostInfoDataForUpdateList)) {
  66. int size = orderAdditionCostInfoDataForUpdateList.size();
  67. int count = size / ;
  68. for (int i = ; i <= count; i++) {
  69. List<OrderAdditionCostInfoDataForUpdate> subList = null;
  70. if (i == count) {
  71. if ( * i < size) {
  72. subList = orderAdditionCostInfoDataForUpdateList.subList( * i, size);
  73. }
  74. } else {
  75. subList = orderAdditionCostInfoDataForUpdateList.subList( * i, * (i + ));
  76. }
  77.  
  78. if (CollectionUtils.isNotEmpty(subList)) {
  79. Future future = dataRefreshJobServiceForAsync.batchUpdateOrderAdditionCostInfoDataForUpdate(subList);
  80. asyncResultList.add(future);
  81. }
  82. }
  83. }
  84. XxlJobLogger.log("批量更新订单附加费数:{}", orderAdditionCostInfoDataForUpdateList.size());
  85.  
  86. if (CollectionUtils.isNotEmpty(asyncResultList)) {
  87. for (Future asyncResult : asyncResultList) {
  88. try {
  89. asyncResult.get();
  90. } catch (Exception e) {
  91. e.printStackTrace();
  92. }
  93. }
  94. }
  95. }

本地库刷新没问题,但是到了线上库就出现如下异常:

  1. ### Error updating database. Cause: org.springframework.jdbc.CannotGetJdbcConnectionException: Failed to obtain JDBC Connection; nested exception is java.sql.SQLTransientConnectionException: HikariPool- - Connection is not available, request timed out after 30005ms.
  2. ### Cause: org.springframework.jdbc.CannotGetJdbcConnectionException: Failed to obtain JDBC Connection; nested exception is java.sql.SQLTransientConnectionException: HikariPool- - Connection is not available, request timed out after 30005ms.
  3. at org.apache.ibatis.exceptions.ExceptionFactory.wrapException(ExceptionFactory.java:)
  4. at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:)
  5. at sun.reflect.GeneratedMethodAccessor380.invoke(Unknown Source)
  6. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:)
  7. at java.lang.reflect.Method.invoke(Method.java:)
  8. at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:)
  9. ... more
  10. Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Failed to obtain JDBC Connection; nested exception is java.sql.SQLTransientConnectionException: HikariPool- - Connection is not available, request timed out after 30005ms.
  11. at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:)
  12. at org.mybatis.spring.transaction.SpringManagedTransaction.openConnection(SpringManagedTransaction.java:)
  13. at org.mybatis.spring.transaction.SpringManagedTransaction.getConnection(SpringManagedTransaction.java:)
  14. at org.apache.ibatis.executor.BaseExecutor.getConnection(BaseExecutor.java:)
  15. at org.apache.ibatis.executor.SimpleExecutor.prepareStatement(SimpleExecutor.java:)
  16. at org.apache.ibatis.executor.SimpleExecutor.doUpdate(SimpleExecutor.java:)
  17. at org.apache.ibatis.executor.BaseExecutor.update(BaseExecutor.java:)
  18. at org.apache.ibatis.executor.CachingExecutor.update(CachingExecutor.java:)
  19. at sun.reflect.GeneratedMethodAccessor381.invoke(Unknown Source)
  20. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:)
  21. at java.lang.reflect.Method.invoke(Method.java:)
  22. at org.apache.ibatis.plugin.Plugin.invoke(Plugin.java:)
  23. at com.sun.proxy.$Proxy521.update(Unknown Source)
  24. at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:)
  25. ... more
  26. Caused by: java.sql.SQLTransientConnectionException: HikariPool- - Connection is not available, request timed out after 30005ms.
  27. at com.zaxxer.hikari.pool.HikariPool.createTimeoutException(HikariPool.java:)
  28. at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:)
  29. at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:)
  30. at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:)
  31. at com.zaxxer.hikari.HikariDataSource$$FastClassBySpringCGLIB$$eeb1ae86.invoke(<generated>)
  32. at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:)
  33. at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:)
  34. at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:)
  35. at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:)
  36. at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:)
  37. at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:)
  38. at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:)
  39. at com.zaxxer.hikari.HikariDataSource$$EnhancerBySpringCGLIB$$f68c05a.getConnection(<generated>)
  40. at org.springframework.jdbc.datasource.DataSourceUtils.fetchConnection(DataSourceUtils.java:)
  41. at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:)
  42. at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:)

二、解决办法

通过异常可以发现是由于获取不到数据库连接导致,猜测是连接数不够的问题,所以修改HikariPool连接池配置,就解决了

修改前:

  1. spring:
  2. datasource:
  3. hikari:
  4. connection-test-query: SELECT FROM DUAL
  5. connection-timeout:
  6. maximum-pool-size:
  7. max-lifetime:
  8. minimum-idle:
  9. connection-init-sql: SET NAMES utf8mb4

修改后:

  1. spring:
  2. datasource:
  3. hikari:
  4. connection-test-query: SELECT FROM DUAL
  5. connection-timeout:
  6. maximum-pool-size:
  7. max-lifetime:
  8. minimum-idle:
  9. validation-timeout:
  10. idle-timeout:
  11. connection-init-sql: SET NAMES utf8mb4

【异常】Caused by: java.sql.SQLTransientConnectionException: HikariPool-1 - Connection is not available, request timed out after 30005ms.的更多相关文章

  1. 异常:Caused by: java.sql.SQLException: Field 'cust_id' doesn't have a default value

    异常: 由Java.q.L.SqLExpExt引起:字段“CuSTyID”没有默认值 Caused by: java.sql.SQLException: Field 'cust_id' doesn't ...

  2. Oracle异常:Caused by: java.sql.SQLException: ORA-01536: 超出表空间 '登录名' 的空间限额 (JPA保存数据)

    原因: Oracle表空间为0,没有分配空间内存. 解决办法在代码框里: 1. 查看用户表空间的限额 select * from user_ts_quotas; max_bytes字段就是了 -1是代 ...

  3. 数据库连接异常 Caused by: java.sql.SQLException: Unknown system variable 'tx_isolation'

    1.错误截图 2.错误分析 数据库的版本比连接驱动的版本高很多. 3.解决方法 因此将mysql-connector-java升级到最新版本就解决了问题. 原本我的版本是mysql-connector ...

  4. hibernate 出现Caused by: java.sql.SQLException: Column 'id' not found.异常

    用hibernate进行映射查询时,出现Caused by: java.sql.SQLException: Column 'id' not found 异常,检查数据库表及映射都有id且已经正确映射, ...

  5. Caused by: java.sql.SQLException: Incorrect integer value: '' for column 'clientId' at row 41

    1.错误描述 [ERROR:]2015-06-10 13:48:26,253 [异常拦截] oa.exception.ExceptionHandler org.hibernate.exception. ...

  6. Caused by: java.sql.SQLException: Parameter index out of range (1 > number of parameters, which is 0

    1.错误描述 [ERROR:]2015-05-05 16:35:50,664 [异常拦截] org.hibernate.exception.GenericJDBCException: error ex ...

  7. Caused by: java.sql.SQLException: Operand should contain 1 column(s)

    1.错误描述 [ERROR:]2015-05-05 15:48:55,847 [异常拦截] org.hibernate.exception.DataException: error executing ...

  8. Caused by: java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@d7c365, see the next exception for details.

    解决方法:https://stackoverflow.com/questions/37442910/spark-shell-startup-errors 异常: 18/01/29 19:04:27 W ...

  9. Caused by: java.sql.BatchUpdateException: Transaction error, need to rollback. errno:1205 Lock wait timeout exceeded; try restarting transaction

    更新的时候报 Caused by: java.sql.BatchUpdateException: Transaction error, need to rollback. errno:1205 Loc ...

随机推荐

  1. django -- ORM实现图书增删改查

    表结构设计 上篇我们实现了出版社的增删改查,出版社数据表有两个字段id和name,那图书的表结构怎么设计呢?图书也要有一个主键id,还要有一个名称title,是哪个出版社的,要有个字段press和Pr ...

  2. CSS3实现PS中的蚁行线动画以及画布的马赛克背景图

    话不多说,先看例子,外链 效果截图如下: 蚁行线 马赛克背景 代码: 蚁行线代码如下: /* <!-- HTML代码 --> <div class="ant"&g ...

  3. vue自学小demo----前端

    vue学习的小demo,实现简单的页面项目的增删 代码如下 <!DOCTYPE html> <html> <head> <meta charset=" ...

  4. 去除批次效应 sva

    Surrogate Variable Analysis http://www.bioconductor.org/packages/release/bioc/html/sva.html

  5. 一些树上dp的复杂度证明

    以下均为内网 树上染色 https://www.lydsy.com/JudgeOnline/problem.php?id=4033 可怜与超市 http://hzoj.com/contest/62/p ...

  6. Keras 中的 verbose 参数

    在 fit( ) 和 evaluate( ) 中 都有 verbose 这个参数,但都是表示日志显示的参数. 具体如下:  fit( ) 中 的 verbose 参数: verbose:日志显示ver ...

  7. unable to find utility "simctl", not a developer tool or in PATH解决方案

    解决方案就是去xcode设置里面,将Command line Tools设置一下,在Xcode>preferences>Locations里面,设置之后再运行终端即可

  8. eclipse中正确创建Django项目

    本教程只说明eclipse中django项目的创建,不涉及django相关开发内容: 1."File" -> "New" -> "Othe ...

  9. python入门之函数对象

    目录 函数是第一类对象 1.函数名可以被引用 2.函数名可以当做参数传递 3.函数名可以当做返回值使用 4.函数名可以被当做容器类型的元素 函数是第一类对象 First-Class Object : ...

  10. angular数据请求 token验证

    angualr的token 验证会经常用在登录,注册等地方 对于token的使用方法按照以下步骤进行使用即可 1.新建一个服务 ng g service services /+服务名  eg:ng g ...