转自:https://www.jeejava.com/spring-data-jpa-batch-insertion/

Spring Data JPA Batch Insertion will show you how we can insert a large dataset into a database at once using Spring Data JPA. For this tutorial we will create a Spring Boot project in Eclipse. We will also see how Spring @Transactional annotation works. Spring transaction required in order to rollback the inserted data at any point if your application fails for any reason.

Sometimes we need to insert or update large number of records in the database. It’s not a good idea to insert multiple records into database one by one in a traditional approach. It will hit the application’s performance. Spring provides batch operations with the help of JpaRepository or CrudRepository, it inserts or updates records into database in one shot. You can also use JDBC API to insert multiple records or batch insertion into database but here we will use Spring JPA’s built-in functionality to get benefits of Spring API.

By default Spring does not save your data into database when you call save() method with multiple entities (a list of objects) passed as argument, hence you have to save entities one by one, which is time consuming and performance gets affected. For this there are few properties that need to be configured to let Spring Data JPA work on batch insertion into database. We will see how these properties set during creation of database configuration class below.

You may also like to read:

Batch Insert using Spring JdbcTemplate

Transaction Management in Spring

Hibernate UserType using Spring Data JPA

Spring EnableEncryptableProperties with Jasypt

Spring Data JPA Entity Auditing using EntityListeners

Spring Data JPA Entity Graph

Spring Data JPA CRUD Example

Prerequisites

The following configurations are required in order to run the application

Eclipse
JDK 1.8
Have gradle installed and configured
Spring dependencies in build.gradle script

Now we will see the below steps how to create a gradle based spring project in Eclipse to work on example Spring Data JPA Batch Insertion.

Creating and setting up the project

First you need to setup the gradle based project in Eclipse and we have to make sure using the below build script we will be able to build the blank project.

In this file notice we have applied required plugins and added required dependencies, such as spring-boot-starter-web, spring-boot-starter-data-jpa and oracle jdbc driver to interact with Java and database API.

Once you create below file, please try to build the project, you should be able to build the blank project.

buildscript {
ext {
springBootVersion = '1.5.9.RELEASE'
}
repositories {
mavenLocal()
mavenCentral()
}
dependencies {
classpath("org.springframework.boot:spring-boot-gradle-plugin:${springBootVersion}")
}
} apply plugin: 'java'
apply plugin: 'org.springframework.boot' sourceCompatibility = 1.8
targetCompatibility = 1.8 repositories {
mavenLocal()
mavenCentral()
} dependencies {
compile('org.springframework.boot:spring-boot-starter-web')
compile("org.springframework.boot:spring-boot-starter-data-jpa")
runtime("com.oracle.jdbc:ojdbc7:12.1.0.2")
}

Creating application.properties file under classpath

Below is the application.properties file under classpath directory src/main/resources and you need to define database credentials to establish connection with database.

Also if you do not want server to run on default port then you may want to specify the server port using server.port key.

Here in Spring Data JPA Batch Insertion example, I am going to use Oracle database but you may use any database as per your requirements.

#datasource
spring.datasource.driverClassName=oracle.jdbc.driver.OracleDriver
spring.datasource.hibernate.dialect=org.hibernate.dialect.Oracle12cDialect
spring.datasource.url=jdbc:Oracle:thin:@//<host>:<port>/<service name>
spring.datasource.username=<username>
spring.datasource.password=<password> server.port=9999 #disable schema generation from Hibernate
spring.jpa.hibernate.ddl-auto=none

Creating database configuration class

Below is the configuration class that will be used to define various database related beans such as DataSource, EntityManagerFactory etc.

As we know JPA is a specification or Interface and someone has to provide its own implementation, so here we are using Hibernate as an implementation of JPA API.

As we have application.properties file in classpath, so we don’t need to load the properties file.

We have let Spring know where our Spring Data JPA Repository interfaces using the annotation @EnableJpaRepositories and we have also let Spring know where to look for Entity classes using the setter method factory.setPackagesToScan(“com.jeejava.entity”).

By default Spring does not work when you want to insert multiple records or entities using save() method of JpaRepository or CrudRepository and that’s why you need to set few properties into

LocalContainerEntityManagerFactoryBean as shown below in entityManagerFactory() method. You can change the batch size, here I have put 500.

By default Spring transaction works out of the box so you may not need to annotate the configuration class with @EnableTransactionManagement.

package com.jeejava.config;

import javax.persistence.EntityManagerFactory;
import javax.sql.DataSource;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.env.Environment;
import org.springframework.data.jpa.repository.config.EnableJpaRepositories;
import org.springframework.jdbc.datasource.DriverManagerDataSource;
import org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean;
import org.springframework.orm.jpa.vendor.Database;
import org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter; @Configuration
@EnableJpaRepositories(basePackages = "com.jeejava.repository")
public class DatabaseConfig { @Autowired
private Environment environment; @Bean
public DataSource dataSource() {
DriverManagerDataSource ds = new DriverManagerDataSource();
ds.setDriverClassName(environment.getRequiredProperty("spring.datasource.driverClassName"));
ds.setUrl(environment.getRequiredProperty("spring.datasource.url"));
ds.setUsername(environment.getRequiredProperty("spring.datasource.username"));
ds.setPassword(environment.getRequiredProperty("spring.datasource.password"));
return ds;
} @Bean
public EntityManagerFactory entityManagerFactory(DataSource dataSource) {
HibernateJpaVendorAdapter vendorAdapter = new HibernateJpaVendorAdapter();
vendorAdapter.setDatabase(Database.ORACLE);
LocalContainerEntityManagerFactoryBean factory = new LocalContainerEntityManagerFactoryBean(); //Use these properties to let spring work on batch insertion
Properties jpaProperties = new Properties();
jpaProperties.put("hibernate.jdbc.batch_size", 500);
jpaProperties.put("hibernate.order_inserts", true);
jpaProperties.put("hibernate.order_updates", true);
lemfb.setJpaProperties(jpaProperties); factory.setJpaVendorAdapter(vendorAdapter);
factory.setPackagesToScan("com.jeejava.entity");
factory.setDataSource(dataSource);
factory.afterPropertiesSet();
return factory.getObject();
}
}

Creating entity class

This is the entity class that maps Java object to database table. This entity class represents a single row in database table. When you want to save multiple rows in database table then you pass a list of entity objects to JpaRepository or CrudRepository’s save() method in order to save multiple entities or objects and this basically happens through Spring Data JPA Batch Insertion configuration. We also save single object using the same save() method.

package com.jeejava.entity;

import java.io.Serializable;
import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.Id;
import javax.persistence.Table; @Entity
@Table(name = "EMPLOYEE")
public class Employee implements Serializable { private static final long serialVersionUID = 1L; @Id
@Column(name = "EMPLOYEE_ID")
private Integer empId; @Column(name = "EMPLOYEE_NAME")
private String empName;
//getters and setters
}

Spring Data JPA Repository

Here is the Spring Data JPA Repository interface. Here JpaRepository takes two parameters Employee object, i.e., entity object and primary key, i.e., Integer. You may have Long, String or any other class object as a primary key as well.

package com.jeejava.repository;

import org.springframework.data.jpa.repository.JpaRepository;
import com.jeejava.entity.Employee; public interface EmployeeRepository extends JpaRepository<Employee, Integer> {
}

Creating Service class

This is the service class that interacts with data layer as well as controller layer and acts as a mediator between them. This class generally handles all business logic.

In this service we will see the example on Spring Data JPA Batch Insertion. Here notice how I am determining when to insert into database.

Here we iterate through list of employee objects and add to temporary Employee array list. Once we find counter equals to batch size(500) then we save those entity objects and at the same time we also clear the temp list because we don’t need those records in temp list any more.

Notice we have used @Transactional annotation in order to support Spring’s transaction management to rollback database insertion at any point of failures.

package com.jeejava.service;

import java.util.List;
import javax.annotation.Resource;
import org.springframework.stereotype.Service;
import com.jeejava.entity.Employee;
import com.jeejava.repository.EmployeeRepository; @Service
public class EmployeeService {
@Resource
private EmployeeRepository employeeRepository;
@Transactional
public void saveEmployees(List<Employee> employees) {
int size = employees.size();
int counter = 0; List<Employee> temp = new ArrayList<>(); for (Employee emp : employees) {
temp.add(emp); if ((counter + 1) % 500 == 0 || (counter + 1) == size) {
employeeRepository.save(temp);
temp.clear();
}
counter++;
}
}
}

Spring REST Controller

The Spring REST Controller class is resposible for handling requests and responses from clients. This holds all the REST services end-points. Using these end-points we would be able to get the JSON response.

Here we have only one end-point called /employees/save that saves a list of employees into database when you hit the URL http://localhost:9999/employees/save from REST client or Postman with a list of employee objects in JSON format as a body parameter.

package com.jeejava.controller;

import java.util.List;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;
import com.jeejava.entity.Employee;
import com.jeejava.service.EmployeeService; @RestController
public class EmployeeRestController { @Autowired
private EmployeeService employeeService; @PostMapping("/employees/save")
public ResponseEntity<Void> saveEmployees(@RequestBody List<Employee> employees) {
employeeService.saveEmployees(employees);
return new ResponseEntity<Void>(HttpStatus.OK);
}
}

Here is the application main class that is enough to start up the application in Spring Boot.

package com.jeejava.application;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication; @SpringBootApplication(scanBasePackages = "com.jeejava")
public class Application { public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
}

Once you run the above main class and application gets start up, hit the URL http://localhost:9999/employees/save from REST client or Postman with list of employee objects as json and you should get the JSON response with OK.

You may also like to read:

Batch Insert using Spring JdbcTemplate

Transaction Management in Spring

Hibernate UserType using Spring Data JPA

Spring EnableEncryptableProperties with Jasypt

Spring Data JPA Entity Auditing using EntityListeners

Spring Data JPA Entity Graph

Spring Data JPA CRUD Example

That’s all. Hope you found idea on Spring Data JPA Batch Insertion.

Thanks for reading.

Spring Data JPA Batch Insertion的更多相关文章

  1. spring data jpa开启批量插入、批量更新

    spring data jpa开启批量插入.批量更新 原文链接:https://www.cnblogs.com/blog5277/p/10661096.html 原文作者:博客园--曲高终和寡 *** ...

  2. Spring Boot:整合Spring Data JPA

    综合概述 JPA是Java Persistence API的简称,是一套Sun官方提出的Java持久化规范.其设计目标主要是为了简化现有的持久化开发工作和整合ORM技术,它为Java开发人员提供了一种 ...

  3. 快速搭建springmvc+spring data jpa工程

    一.前言 这里简单讲述一下如何快速使用springmvc和spring data jpa搭建后台开发工程,并提供了一个简单的demo作为参考. 二.创建maven工程 http://www.cnblo ...

  4. spring boot(五):spring data jpa的使用

    在上篇文章springboot(二):web综合开发中简单介绍了一下spring data jpa的基础性使用,这篇文章将更加全面的介绍spring data jpa 常见用法以及注意事项 使用spr ...

  5. 转:使用 Spring Data JPA 简化 JPA 开发

    从一个简单的 JPA 示例开始 本文主要讲述 Spring Data JPA,但是为了不至于给 JPA 和 Spring 的初学者造成较大的学习曲线,我们首先从 JPA 开始,简单介绍一个 JPA 示 ...

  6. 深入浅出学Spring Data JPA

    第一章:Spring Data JPA入门 Spring Data是什么 Spring Data是一个用于简化数据库访问,并支持云服务的开源框架.其主要目标是使得对数据的访问变得方便快捷,并支持map ...

  7. spring data jpa 调用存储过程

    网上这方面的例子不是很多,研究了一下,列出几个调用的方法. 假如我们有一个mysql的存储过程 CREATE DEFINER=`root`@`localhost` PROCEDURE `plus1in ...

  8. Spring Data JPA 学习记录1 -- 单向1:N关联的一些问题

    开新坑 开新坑了(笑)....公司项目使用的是Spring Data JPA做持久化框架....学习了一段时间以后发现了一点值得注意的小问题.....与大家分享 主要是针对1:N单向关联产生的一系列问 ...

  9. Spring Boot with Spring Data JPA (1) - Concept

    What's Spring Data JPA? According to Pivotal, Spring Data JPA, part of the larger Spring Data family ...

随机推荐

  1. java程序员一些初中级面试题(数据库部分)

    说出一些数据库优化方面的经验? 1.从JDBC编程的角度讲,用PreparedStatement一般来说比Statement性能高,因为在使用时,SQL语句被预编译并存储在PreparedStatem ...

  2. [转帖]cmd批处理常用符号详解

    cmd批处理常用符号详解 https://www.jb51.net/article/32866.htm 很多符号 还是不清楚的.. 批处理能够极大的提高 工作效率 需要加强深入学习.   1.@一般在 ...

  3. Day 3-4 函数进阶

    1.名称空间 定义:Python使用叫做命名空间的东西来记录变量的轨迹.命名空间是一个 字典(dictionary) ,它的键就是变量名,它的值就是那些变量的值.是存放变量和值的内存地址的绑定关系的空 ...

  4. CLOUD设置过滤方案不共享

    1.打开BOS,找到应用框架-动态表单-过滤方案另存 2.找到共享给他人,把可见性全部去掉

  5. Field tTypeMapper in com.atguigu.project.service.imp.projectInfoServiceImpl required a bean of type 'com.atguigu.project.mapper.TTypeMapper' that could not be found.

    解决:MapperScan

  6. Django--CRM--一级, 二级 菜单表

    一. 一级菜单表 1. 首先要修改权限表的字段, 在权限表下面加上icon和 is_menu 的字段 2. 展示结果 # 我们既然想要动态生成一级菜单,那么就需要从数据库中拿出当前登录的用户的菜单表是 ...

  7. MySQL执行语句的顺序

    MySQL的语句一共分为11步,最先执行的总是FROM操作,最后执行的是LIMIT操作.其中每一个操作都会产生一张虚拟的表,这个虚拟的表作为一个处理的输入,只是这些虚拟的表对用户来说是透明的,但是只有 ...

  8. layui tips

  9. luogu1503

    P1503 鬼子进村 题目背景 小卡正在新家的客厅中看电视.电视里正在播放放了千八百次依旧重播的<亮剑>,剧中李云龙带领的独立团在一个县城遇到了一个鬼子小队,于是独立团与鬼子展开游击战. ...

  10. 洛谷 P2119 魔法阵

    题目描述 六十年一次的魔法战争就要开始了,大魔法师准备从附近的魔法场中汲取魔法能量. 大魔法师有mm个魔法物品,编号分别为1,2,...,m1,2,...,m.每个物品具有一个魔法值,我们用X_iXi ...