目录

  一、Sharding-JDBC依赖

  二、代码实践

  三、源码分析

在上一篇博文中,介绍了Sharding-JDBC的分片策略、分片键和分片算法的基本概念,以及2.0.3版本可以支持和无法支持的使用场景。

可以支持的场景:支持对SQL语句中的=、IN和BETWEEN AND的分片操作,但前提是分片键必须存在于SQL和数据表结构中。

无法支持的场景:分片键不存在于SQL和数据表结构中,即基于暗示(Hint)的数据分片操作(2.0.3版本的问题)。

无可厚非,缺少了Hint分片策略的支持,Sharding-JDBC 2.0.3版本的使用场景就非常受限了,但值得庆幸的是,此问题在3.x版本进行了修复(这里可以有掌声!),接下来的代码皆基于3.1.0版本。

一、Sharding-JDBC依赖

<!-- sharding-jdbc-core -->
<dependency>
<groupId>io.shardingsphere</groupId>
<artifactId>sharding-jdbc-core</artifactId>
<version>3.1.0</version>
</dependency>
<!-- sharding-jdbc-spring-namespace -->
<dependency>
<groupId>io.shardingsphere</groupId>
<artifactId>sharding-jdbc-spring-namespace</artifactId>
<version>3.1.0</version>
</dependency>

和2.0.3版本相比,依赖的名称有所改变,不要搞错了哦。

二、代码实践

业务背景就不再介绍了,不了解可移步至Sharding-JDBC(二)2.0.3版本实践

如下代码配置了标准分片策略中的精确分片算法PreciseShardingAlgorithm和Hint分片算法HintShardingAlgorithm。

XML配置:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:tx="http://www.springframework.org/schema/tx"
xmlns:sharding="http://shardingsphere.io/schema/shardingsphere/sharding"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/tx
http://www.springframework.org/schema/tx/spring-tx.xsd
http://shardingsphere.io/schema/shardingsphere/sharding
http://shardingsphere.io/schema/shardingsphere/sharding/sharding.xsd"> <!-- 标准分片策略 -->
<sharding:standard-strategy id="settlementTableShardingStandardStrategy" sharding-column="pay_serial_number"
precise-algorithm-ref="preciseTableShardingAlgorithm"/> <!-- 基于暗示(Hint)的分片策略 -->
<sharding:hint-strategy id="settlementHintTableShardingStrategy" algorithm-ref="hintTableShardingAlgorithm"/>
<sharding:hint-strategy id="settlementHintDatabaseShardingStrategy" algorithm-ref="hintDatabaseShardingAlgorithm"/> <sharding:data-source id="shardingDataSource">
<sharding:sharding-rule data-source-names="dataSource">
<sharding:table-rules>
<sharding:table-rule logic-table="settlement"
table-strategy-ref="settlementTableShardingStandardStrategy"/>
<!-- logic-table参数的大小写必须和SettlementMapper.xml中selectByExample方法的表名大小一致!!! -->
<!-- logic-table必须和org.cellphone.finance.repo.SettlementRepository.querySettlements中的logicTable及SQL中的表名一致,否则无法找到分片策略 -->
<!-- 逻辑表名,不需要和真实表名一致 -->
<sharding:table-rule logic-table="settlement_hint"
database-strategy-ref="settlementHintDatabaseShardingStrategy"
table-strategy-ref="settlementHintTableShardingStrategy"/>
</sharding:table-rules>
</sharding:sharding-rule>
<sharding:props>
<prop key="sql.show">true</prop>
</sharding:props>
</sharding:data-source> <bean id="transactionManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
<property name="dataSource" ref="shardingDataSource"/>
</bean>
<tx:annotation-driven/> </beans>

精确分片算法:

package org.cellphone.finance.repo.sharding;

import com.google.common.collect.Lists;
import io.shardingsphere.api.algorithm.sharding.PreciseShardingValue;
import io.shardingsphere.api.algorithm.sharding.standard.PreciseShardingAlgorithm;
import org.apache.commons.collections.CollectionUtils;
import org.cellphone.common.constant.CommonConst;
import org.springframework.stereotype.Component; import java.util.Collection; /**
* 精确分片算法,属于标准分片算法,用于处理=和IN的分片
* <p>
* 使用精确分片算法的前提:分片字段必须存在与SQL中、数据库表结构中,否则无法使用精确分片算法
* <p>
* 此分片算法应用于SETTLEMENT数据表,这里是按天分表
* <p>
* 特别说明:Sharding Jdbc版本:3.1.0
* <p>
* Created by on 2018/4/9.
*/
@Component("preciseTableShardingAlgorithm")
public class PreciseTableShardingAlgorithm implements PreciseShardingAlgorithm<String> { /**
* 精确分片算法
*
* @param availableTargetNames 目标数据源名称或数据表名称,注意:是逻辑数据源名或逻辑数据表名,来自SQL
* @param shardingValue 分片值,来自SQL中分片字段对应的值
* @return 真实数据源名称或数据表名称
*/
@Override
public String doSharding(final Collection<String> availableTargetNames, final PreciseShardingValue<String> shardingValue) {
// 默认数据表名称,有可能数据库中不存在这张表
String tableName = "settlement"; // 逻辑表名为空,返回默认表名
if (CollectionUtils.isEmpty(availableTargetNames))
return tableName; // availableTargetNames来自SQL,只有一个元素
tableName = Lists.newArrayList(availableTargetNames).get(0); String paySerialNumber = shardingValue.getValue();
String suffix = paySerialNumber.substring(5, 13);
return tableName + CommonConst.UNDERLINE + suffix;
}
}

Hint数据源分片算法:

package org.cellphone.finance.repo.sharding;

import io.shardingsphere.api.algorithm.sharding.ShardingValue;
import io.shardingsphere.api.algorithm.sharding.hint.HintShardingAlgorithm;
import org.springframework.stereotype.Component; import java.util.Collection; /**
* Sharding Jdbc基于暗示(Hint)的数据分片算法
*
* 使用Sharding Jdbc 3.x版本时,此数据源分片算法这个一定要有!!!
* 否则无法正常使用org.cellphone.finance.repo.sharding.HintTableShardingAlgorithm算法
* <p>
* Created by on 2019/4/25.
*/
@Component("hintDatabaseShardingAlgorithm")
public class HintDatabaseShardingAlgorithm implements HintShardingAlgorithm { @Override
public Collection<String> doSharding(Collection<String> availableTargetNames, ShardingValue shardingValue) {
return availableTargetNames;
}
}

Hint数据表分片算法:

package org.cellphone.finance.repo.sharding;

import com.google.common.collect.Lists;
import io.shardingsphere.api.algorithm.sharding.ListShardingValue;
import io.shardingsphere.api.algorithm.sharding.ShardingValue;
import io.shardingsphere.api.algorithm.sharding.hint.HintShardingAlgorithm;
import org.apache.commons.collections.CollectionUtils;
import org.apache.commons.lang3.StringUtils;
import org.apache.commons.lang3.time.DateFormatUtils;
import org.apache.commons.lang3.time.DateUtils;
import org.cellphone.common.constant.CommonConst;
import org.cellphone.common.constant.DateConst;
import org.springframework.stereotype.Component; import java.text.ParseException;
import java.util.*; /**
* Sharding Jdbc基于暗示(Hint)的数据分片算法
* 版本:Sharding Jdbc 3.1.0
*
* 官方介绍(2.x版本):http://shardingsphere.apache.org/document/legacy/2.x/cn/02-guide/hint-sharding-value/
* 官方介绍(4.x版本):https://shardingsphere.apache.org/document/current/cn/manual/sharding-jdbc/usage/hint/
* <p>
* <p>
* <p>
* <p>
* 使用此算法的背景如下:
* 1. SETTLEMENT(支付表)是按时间维度进行分表,该时间取自PAY_SERIAL_NUMBER中的时间数据,非表中的时间字段;
* <p>
* 2. 使用用户手机号此类条件查询数据时,请求参数除传入业务参数外,
* 还需传入时间段(即分片字段,例:startTime和endTime)以确定分表范围;
* 但是!!!startTime和endTime(即分片字段)不存在SQL中、数据库表结构中,而存在于外部业务逻辑
* <p>
* <p>
* <p>
* 因此,第2点导致无法直接使用Sharding Jdbc的PreciseShardingAlgorithm(精确分片算法)或RangeShardingAlgorithm(范围分片算法),
* 只能使用HintShardingAlgorithm(基于暗示的数据分片算法),该算法的使用场景如下:
* 1. 分片字段不存在SQL中、数据库表结构中,而存在于外部业务逻辑;
* <p>
* 2. 强制在主库进行某些数据操作。
* <p>
* <p>
* Created by on 2019/4/25.
*/
@Component("hintTableShardingAlgorithm")
public class HintTableShardingAlgorithm implements HintShardingAlgorithm { /**
* 分片算法
*
* @param availableTargetNames 逻辑数据库名称或逻辑数据表名称
* @param shardingValue 用来确定分表的参数
* @return 实际数据表名称列表,SQL实际操作的数据表
*/
@Override
public Collection<String> doSharding(Collection<String> availableTargetNames, ShardingValue shardingValue) { String realTableName = StringUtils.EMPTY;
for (String each : availableTargetNames) {
if (StringUtils.isNotBlank(each)) {
// 基于hint的逻辑表名:settlement_hint
realTableName = each.replace("_hint", StringUtils.EMPTY);
break;
}
} List<String> tables = new ArrayList<>(); ListShardingValue<String> listShardingValue = (ListShardingValue<String>) shardingValue;
List<String> list = Lists.newArrayList(listShardingValue.getValues()); // 缺少确定分表的参数,无法确定具体分表,直接返回真实表名称
if (CollectionUtils.isEmpty(list)) {
tables.add(realTableName);
return tables;
} // 拆分分表参数,此参数值来自:com.fcbox.manage.core.repo.FcBoxPostRepository.queryFcBoxPosts()
String[] queryTime = list.get(0).split(CommonConst.UNDERLINE);
Date startTime, endTime;
try {
startTime = DateUtils.parseDate(queryTime[0], DateConst.DATE_FORMAT_NORMAL);
endTime = DateUtils.parseDate(queryTime[1], DateConst.DATE_FORMAT_NORMAL);
} catch (ParseException e) {
// 分表参数解析错误,无法确定具体分表,直接返回真实表名称
tables.add(realTableName);
return tables;
} Calendar calendar = Calendar.getInstance();
// 组织startTime和endTime时段范围内的分表
while (startTime.getTime() <= endTime.getTime()) {
tables.add(realTableName + CommonConst.UNDERLINE + DateFormatUtils.format(startTime, DateConst.DATE_FORMAT_YYYY_MM_DD));
calendar.setTime(startTime);
calendar.add(Calendar.DATE, 1);
startTime = calendar.getTime();
} return tables;
}
}

与Hint分片算法对应的Java查询方法 settlementMapper.selectByExample(example):

    public List<Settlement> querySettlements(SettlementExample example, String startTime, String endTime) {
// 组织查询时间,传入org.cellphone.finance.repo.sharding.HintTableShardingAlgorithm分片算法中以确认具体分表
String queryTime = startTime + CommonConst.UNDERLINE + endTime; // 获取HintManager
HintManager hintManager = HintManager.getInstance();
/*
* 添加数据源分片键值,使用Sharding Jdbc 3.x版本一定要添加数据源分片键值,否则无法使用HintTableShardingAlgorithm分片算法
* 若无分库,addDatabaseShardingValue方法的value字段随意填充
* 若有分库,addDatabaseShardingValue方法的value字段填充实际参数值
*/
hintManager.addDatabaseShardingValue("settlement_hint", StringUtils.EMPTY);
// 添加数据表分片键值
hintManager.addTableShardingValue("settlement_hint", queryTime);
List<Settlement> settlements = settlementMapper.selectByExample(example);
// 清除分片键值
hintManager.close();
return settlements;
}

以及该查询方法对应的SQL语句:

select * from settlement_hint t where t.pay_serial_number = ?

单元测试代码:

@Test
public void test003QuerySettlements() throws ParseException {
String startTime = "2018-04-03 00:00:00", endTime = "2018-04-05 00:00:00"; SettlementExample example = new SettlementExample();
SettlementExample.Criteria criteria = example.createCriteria();
criteria.andPaySerialNumberEqualTo(paySerialNumber); List<Settlement> settlements = repository.querySettlements(example, startTime, endTime); Assert.assertEquals("136********", settlements.get(0).getUserMobile());
}

三、源码分析

和2.0.3版本相比,3.1.0版本的路由入口变成了 io.shardingsphere.core.routing.type.standard.StandardRoutingEngine#route() ,但基本上区别不大,仅仅是多了一步需要确定路由的真实数据源,尽管数据源只有一个,也需要显式配置数据源路由算法。代码中标注了注释的部分都是路由代码比较核心的部分。

/*
* Copyright 2016-2018 shardingsphere.io.
* <p>
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
* </p>
*/ package io.shardingsphere.core.routing.type.standard; import com.google.common.base.Optional;
import com.google.common.base.Preconditions;
import io.shardingsphere.api.algorithm.sharding.ShardingValue;
import io.shardingsphere.core.hint.HintManagerHolder;
import io.shardingsphere.core.optimizer.condition.ShardingCondition;
import io.shardingsphere.core.optimizer.condition.ShardingConditions;
import io.shardingsphere.core.optimizer.insert.InsertShardingCondition;
import io.shardingsphere.core.routing.strategy.ShardingStrategy;
import io.shardingsphere.core.routing.strategy.hint.HintShardingStrategy;
import io.shardingsphere.core.routing.type.RoutingEngine;
import io.shardingsphere.core.routing.type.RoutingResult;
import io.shardingsphere.core.routing.type.RoutingTable;
import io.shardingsphere.core.routing.type.TableUnit;
import io.shardingsphere.core.rule.BindingTableRule;
import io.shardingsphere.core.rule.DataNode;
import io.shardingsphere.core.rule.ShardingRule;
import io.shardingsphere.core.rule.TableRule;
import lombok.RequiredArgsConstructor; import java.util.ArrayList;
import java.util.Collection;
import java.util.Collections;
import java.util.LinkedHashSet;
import java.util.LinkedList;
import java.util.List; /**
* Standard routing engine.
*
* @author zhangliang
* @author maxiaoguang
* @author panjuan
*/
@RequiredArgsConstructor
public final class StandardRoutingEngine implements RoutingEngine { private final ShardingRule shardingRule; private final String logicTableName; private final ShardingConditions shardingConditions; /**
* 相比2.0.3版本,精简成了1行代码
*
* @return 路由结果
*/
@Override
public RoutingResult route() {
return generateRoutingResult(getDataNodes(shardingRule.getTableRuleByLogicTableName(logicTableName)));
} private RoutingResult generateRoutingResult(final Collection<DataNode> routedDataNodes) {
RoutingResult result = new RoutingResult();
for (DataNode each : routedDataNodes) {
TableUnit tableUnit = new TableUnit(each.getDataSourceName());
tableUnit.getRoutingTables().add(new RoutingTable(logicTableName, each.getTableName()));
result.getTableUnits().getTableUnits().add(tableUnit);
}
return result;
} /**
* 获取数据节点,即真实表
*
* @param tableRule 从XML读取的table rule
* @return 数据节点列表
*/
private Collection<DataNode> getDataNodes(final TableRule tableRule) {
// 判断是否是通过Hint分片策略进行路由
if (isRoutingByHint(tableRule)) {
return routeByHint(tableRule);
}
if (isRoutingByShardingConditions(tableRule)) {
return routeByShardingConditions(tableRule);
}
return routeByMixedConditions(tableRule);
} /**
* 判断是否是通过Hint分片策略进行路由
* <p>
* 数据源分片策略和数据表分片策略都必须是HintShardingStrategy,这意味者必须显式配置数据源Hint分片策略和数据表Hint分片策略
*
* @param tableRule 从XML读取的table rule
* @return
*/
private boolean isRoutingByHint(final TableRule tableRule) {
return shardingRule.getDatabaseShardingStrategy(tableRule) instanceof HintShardingStrategy && shardingRule.getTableShardingStrategy(tableRule) instanceof HintShardingStrategy;
} /**
* 通过Hint分片策略进行路由
*
* @param tableRule
* @return
*/
private Collection<DataNode> routeByHint(final TableRule tableRule) {
return route(tableRule, getDatabaseShardingValuesFromHint(), getTableShardingValuesFromHint());
} private boolean isRoutingByShardingConditions(final TableRule tableRule) {
return !(shardingRule.getDatabaseShardingStrategy(tableRule) instanceof HintShardingStrategy || shardingRule.getTableShardingStrategy(tableRule) instanceof HintShardingStrategy);
} private Collection<DataNode> routeByShardingConditions(final TableRule tableRule) {
return shardingConditions.getShardingConditions().isEmpty() ? route(tableRule, Collections.<ShardingValue>emptyList(), Collections.<ShardingValue>emptyList())
: routeByShardingConditionsWithCondition(tableRule);
} private Collection<DataNode> routeByShardingConditionsWithCondition(final TableRule tableRule) {
Collection<DataNode> result = new LinkedList<>();
for (ShardingCondition each : shardingConditions.getShardingConditions()) {
Collection<DataNode> dataNodes = route(tableRule, getShardingValuesFromShardingConditions(shardingRule.getDatabaseShardingStrategy(tableRule).getShardingColumns(), each),
getShardingValuesFromShardingConditions(shardingRule.getTableShardingStrategy(tableRule).getShardingColumns(), each));
reviseShardingConditions(each, dataNodes);
result.addAll(dataNodes);
}
return result;
} private Collection<DataNode> routeByMixedConditions(final TableRule tableRule) {
return shardingConditions.getShardingConditions().isEmpty() ? routeByMixedConditionsWithHint(tableRule) : routeByMixedConditionsWithCondition(tableRule);
} private Collection<DataNode> routeByMixedConditionsWithCondition(final TableRule tableRule) {
Collection<DataNode> result = new LinkedList<>();
for (ShardingCondition each : shardingConditions.getShardingConditions()) {
Collection<DataNode> dataNodes = route(tableRule, getDatabaseShardingValues(tableRule, each), getTableShardingValues(tableRule, each));
reviseShardingConditions(each, dataNodes);
result.addAll(dataNodes);
}
return result;
} private Collection<DataNode> routeByMixedConditionsWithHint(final TableRule tableRule) {
if (shardingRule.getDatabaseShardingStrategy(tableRule) instanceof HintShardingStrategy) {
return route(tableRule, getDatabaseShardingValuesFromHint(), Collections.<ShardingValue>emptyList());
}
return route(tableRule, Collections.<ShardingValue>emptyList(), getTableShardingValuesFromHint());
} private List<ShardingValue> getDatabaseShardingValues(final TableRule tableRule, final ShardingCondition shardingCondition) {
ShardingStrategy dataBaseShardingStrategy = shardingRule.getDatabaseShardingStrategy(tableRule);
return isGettingShardingValuesFromHint(dataBaseShardingStrategy)
? getDatabaseShardingValuesFromHint() : getShardingValuesFromShardingConditions(dataBaseShardingStrategy.getShardingColumns(), shardingCondition);
} private List<ShardingValue> getTableShardingValues(final TableRule tableRule, final ShardingCondition shardingCondition) {
ShardingStrategy tableShardingStrategy = shardingRule.getTableShardingStrategy(tableRule);
return isGettingShardingValuesFromHint(tableShardingStrategy)
? getTableShardingValuesFromHint() : getShardingValuesFromShardingConditions(tableShardingStrategy.getShardingColumns(), shardingCondition);
} private boolean isGettingShardingValuesFromHint(final ShardingStrategy shardingStrategy) {
return shardingStrategy instanceof HintShardingStrategy;
} /**
* 从HintManagerHolder中获取数据源分片值
*
* @return 数据源分片值列表
*/
private List<ShardingValue> getDatabaseShardingValuesFromHint() {
// getDatabaseShardingValue方法实现有点恶心,不兼容大小写...
Optional<ShardingValue> shardingValueOptional = HintManagerHolder.getDatabaseShardingValue(logicTableName);
return shardingValueOptional.isPresent() ? Collections.singletonList(shardingValueOptional.get()) : Collections.<ShardingValue>emptyList();
} private List<ShardingValue> getTableShardingValuesFromHint() {
// getTableShardingValue方法实现有点恶心,不兼容大小写...
Optional<ShardingValue> shardingValueOptional = HintManagerHolder.getTableShardingValue(logicTableName);
return shardingValueOptional.isPresent() ? Collections.singletonList(shardingValueOptional.get()) : Collections.<ShardingValue>emptyList();
} private List<ShardingValue> getShardingValuesFromShardingConditions(final Collection<String> shardingColumns, final ShardingCondition shardingCondition) {
List<ShardingValue> result = new ArrayList<>(shardingColumns.size());
for (ShardingValue each : shardingCondition.getShardingValues()) {
Optional<BindingTableRule> bindingTableRule = shardingRule.findBindingTableRule(logicTableName);
if ((logicTableName.equals(each.getLogicTableName()) || bindingTableRule.isPresent() && bindingTableRule.get().hasLogicTable(logicTableName))
&& shardingColumns.contains(each.getColumnName())) {
result.add(each);
}
}
return result;
} /**
* 路由,获取真实表列表
*
* @param tableRule 从XML读取的table rule
* @param databaseShardingValues 数据源分片值
* @param tableShardingValues 数据表分片值
* @return 真实表列表
*/
private Collection<DataNode> route(final TableRule tableRule, final List<ShardingValue> databaseShardingValues, final List<ShardingValue> tableShardingValues) {
Collection<String> routedDataSources = routeDataSources(tableRule, databaseShardingValues);
Collection<DataNode> result = new LinkedList<>();
for (String each : routedDataSources) {
result.addAll(routeTables(tableRule, each, tableShardingValues));
}
return result;
} /**
* 路由到真实数据源
*
* @param tableRule 从XML读取的table rule
* @param databaseShardingValues 数据源分片值
* @return 真实数据源列表
*/
private Collection<String> routeDataSources(final TableRule tableRule, final List<ShardingValue> databaseShardingValues) {
Collection<String> availableTargetDatabases = tableRule.getActualDatasourceNames();
if (databaseShardingValues.isEmpty()) {
return availableTargetDatabases;
}
Collection<String> result = new LinkedHashSet<>(shardingRule.getDatabaseShardingStrategy(tableRule).doSharding(availableTargetDatabases, databaseShardingValues));
Preconditions.checkState(!result.isEmpty(), "no database route info");
return result;
} /**
* 路由到真实数据表,和2.0.3版本没啥区别
*
* @param tableRule 从XML读取的table rule
* @param routedDataSource 已确认好的数据源
* @param tableShardingValues 数据表分片值
* @return 真实表列表
*/
private Collection<DataNode> routeTables(final TableRule tableRule, final String routedDataSource, final List<ShardingValue> tableShardingValues) {
Collection<String> availableTargetTables = tableRule.getActualTableNames(routedDataSource);
Collection<String> routedTables = new LinkedHashSet<>(tableShardingValues.isEmpty() ? availableTargetTables
: shardingRule.getTableShardingStrategy(tableRule).doSharding(availableTargetTables, tableShardingValues));
Preconditions.checkState(!routedTables.isEmpty(), "no table route info");
Collection<DataNode> result = new LinkedList<>();
for (String each : routedTables) {
result.add(new DataNode(routedDataSource, each));
}
return result;
} private void reviseShardingConditions(final ShardingCondition each, final Collection<DataNode> dataNodes) {
if (each instanceof InsertShardingCondition) {
((InsertShardingCondition) each).getDataNodes().addAll(dataNodes);
}
}
}

分析到此,Sharding-JDBC 3.1.0版本可以支持分片键不存在于SQL中和数据表结构中的使用场景。但3.1.0版本还有一个比较恶心的地方,Sharding-JDBC在初始化时,会连接数据库获取数据表的元数据,包括需要水平切分的表和不需水平切分的表。代码如下:

/*
* Copyright 2016-2018 shardingsphere.io.
* <p>
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
* </p>
*/ package io.shardingsphere.core.metadata.table.executor; import com.google.common.base.Optional;
import io.shardingsphere.core.exception.ShardingException;
import io.shardingsphere.core.executor.ShardingExecuteEngine;
import io.shardingsphere.core.metadata.datasource.DataSourceMetaData;
import io.shardingsphere.core.metadata.datasource.ShardingDataSourceMetaData;
import io.shardingsphere.core.metadata.table.TableMetaData;
import io.shardingsphere.core.rule.ShardingRule;
import io.shardingsphere.core.rule.TableRule; import java.sql.Connection;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.util.Collection;
import java.util.HashMap;
import java.util.LinkedHashSet;
import java.util.Map; /**
* Table meta data initializer.
*
* @author zhangliang
*/
public final class TableMetaDataInitializer { private final ShardingDataSourceMetaData shardingDataSourceMetaData; private final TableMetaDataConnectionManager connectionManager; private final TableMetaDataLoader tableMetaDataLoader; public TableMetaDataInitializer(final ShardingDataSourceMetaData shardingDataSourceMetaData, final ShardingExecuteEngine executeEngine,
final TableMetaDataConnectionManager connectionManager, final int maxConnectionsSizePerQuery, final boolean isCheckingMetaData) {
this.shardingDataSourceMetaData = shardingDataSourceMetaData;
this.connectionManager = connectionManager;
tableMetaDataLoader = new TableMetaDataLoader(shardingDataSourceMetaData, executeEngine, connectionManager, maxConnectionsSizePerQuery, isCheckingMetaData);
} /**
* Load all table meta data.
*
* @param shardingRule sharding rule
* @return all table meta data
*/
public Map<String, TableMetaData> load(final ShardingRule shardingRule) {
Map<String, TableMetaData> result = new HashMap<>();
try {
// 加载需要水平切分的表元数据
result.putAll(loadShardingTables(shardingRule));
// 加载不需水平切分的表元数据,如果数据库中表数量很大,这里耗时很久...
result.putAll(loadDefaultTables(shardingRule));
} catch (final SQLException ex) {
throw new ShardingException(ex);
}
return result;
} private Map<String, TableMetaData> loadShardingTables(final ShardingRule shardingRule) throws SQLException {
Map<String, TableMetaData> result = new HashMap<>(shardingRule.getTableRules().size(), 1);
for (TableRule each : shardingRule.getTableRules()) {
result.put(each.getLogicTable(), tableMetaDataLoader.load(each.getLogicTable(), shardingRule));
}
return result;
} private Map<String, TableMetaData> loadDefaultTables(final ShardingRule shardingRule) throws SQLException {
Map<String, TableMetaData> result = new HashMap<>(shardingRule.getTableRules().size(), 1);
Optional<String> actualDefaultDataSourceName = shardingRule.findActualDefaultDataSourceName();
if (actualDefaultDataSourceName.isPresent()) {
for (String each : getAllTableNames(actualDefaultDataSourceName.get())) {
result.put(each, tableMetaDataLoader.load(each, shardingRule));
}
}
return result;
} /**
* 连接数据库,获取所有表元数据
*
* @param dataSourceName 数据源名称
* @return 所有数据表元数据列表
* @throws SQLException
*/
private Collection<String> getAllTableNames(final String dataSourceName) throws SQLException {
Collection<String> result = new LinkedHashSet<>();
DataSourceMetaData dataSourceMetaData = shardingDataSourceMetaData.getActualDataSourceMetaData(dataSourceName);
String catalog = null == dataSourceMetaData ? null : dataSourceMetaData.getSchemeName();
try (Connection connection = connectionManager.getConnection(dataSourceName);
ResultSet resultSet = connection.getMetaData().getTables(catalog, null, null, new String[]{"TABLE"})) {
while (resultSet.next()) {
String tableName = resultSet.getString("TABLE_NAME");
if (!tableName.contains("$")) {
result.add(tableName);
}
}
}
return result;
}
}

Sharding-JDBC(三)3.1.0版本实践的更多相关文章

  1. sharding jdbc(sphere) 3.1.0 spring boot配置

    sharding jdbc 2.x系列详解参见https://www.cnblogs.com/zhjh256/p/9221634.html. 最近将sharding jdbc的配置从xml切换到了sp ...

  2. MySQL的JDBC驱动(8.0版本)

    1.引用外部库  mysql-connector-java-8.0.版本的jar 2.jdbc驱动类:com.mysql.jdbc.Driver  改成 com.mysql.cj.jdbc.Drive ...

  3. FlinkCDC 2.0使用实践体验

    一.背景说明 所谓CDC:全称是 Change Data Capture ,在广义的概念上,只要能捕获数据变更的技术,我们都可以称为 CDC .通常我们说的 CDC 技术主要面向数据库的变更,是一种用 ...

  4. Sharding-JDBC(二)2.0.3版本实践

    目录 一.Sharding-JDBC依赖 二.分片策略 1. 标准分片策略 2. 复合分片策略 3. Inline表达式分片策略 4. 通过Hint而非SQL解析的方式分片的策略 5. 不分片的策略 ...

  5. 使用8.0版本jdbc驱动连接数据库操作

    转自:https://blog.csdn.net/qq_41734885/article/details/80562815 首先,我居然不能用navicat客户端连接上mysql8.0数据库报1251 ...

  6. 三、主流区块链技术特点及Hyperledger Fabric V1.0版本特点

    一.Hyperledger fabric V1.0 架构 1.逻辑架构: 2.区块链网络 3.运行时架构 二.架构总结 1.架构要点 分拆Peer的功能,将Blockchain的数据维护和共识服务进行 ...

  7. ElasticSearch5.0+版本分词热更新实践记录

    前言 刚开始接触ElasticSearch的时候,版本才是2.3.4,短短的时间,现在都更新到5.0+版本了.分词和head插件好像用法也不一样了,本博客记录如何配置Elasticsearch的Hea ...

  8. mysql 数据库8.0版本,jdbc驱动连接问题

    前言 8.0版本的mysql数据的连接 与 5.0的有所不同,下面直接贴出  8.0版本应该有的 jdbc驱动连接,还有 mysql 的jdbc jar包要8.0以上的 内容如下 : jdbc.dri ...

  9. Sharding jdbc 强制路由策略(HintShardingStrategy)使用记录

    背景 随着项目运行时间逐渐增加,数据库中的数据也越来越多,虽然加索引,优化查询,但是数据量太大,还是会影响查询效率,也给数据库增加了负载. 再加上冷数据基本不使用的场景,决定采用分表来处理数据,从而来 ...

随机推荐

  1. MinGW ,GNU 是什么

    MinGW : Minimalist GNU for Windows MinGW(Minimalist GNU For Windows)是个精简的Windows平台下的 C/C++.ADA及Fortr ...

  2. 【python小记】访问mysql数据库

    题记: 最近因为工作需要,学习了python,瞬间对这个轻松快捷的语给吸引了,以前只知道js脚本是写网页的,没有想到python这个脚本语言的应用范围可以这么广泛,现在做一些简单或稍微复杂的操作,基本 ...

  3. 函数的练习1——python编程从入门到实践

    8-1 消息: 编写一个名为display_message()的函数,它打印一个句子指出你在本章学的是什么.调用这个函数,确认显示的消息正确无误. def display_message(): pri ...

  4. DjangoRestFramework学习一之restful规范、APIview、解析器组件、Postman等

    DjangoRestFramework学习一之restful规范.APIview.解析器组件.Postman等 本节目录 一 预备知识 二 restful规范 三 DRF的APIView和解析器组件 ...

  5. day34——僵尸进程和孤儿进程、互斥锁、进程之间的通信

    day34 僵尸进程和孤儿进程 基于unix环境(linux,macOS) 主进程需要等待子进程结束之后,主进程才结束 主进程时刻监测子进程的运行状态,当子进程结束之后,一段时间之内,将子进程进行回收 ...

  6. 1. RDD概念

    1.1 RDD为什么会产生? RDD 是 Spark 的基石,是实现 Spark 数据处理的核心抽象.那么 RDD 为 什么会产生呢? Hadoop 的 MapReduce 是一种基于数据集的工作模式 ...

  7. vscode 连接远程服务器 sftp

    1.在vscode 应用商店搜索 sftp 下载安装 2.ctrl+shift+p 搜索sftp:config 生成sftp.json 3.配置你的服务器信息{ "name": & ...

  8. JavaNetty心跳监控

    import java.net.InetAddress; import java.net.UnknownHostException; import java.util.Map; import java ...

  9. 运行时找到main方法所在的类

    private Class<?> deduceMainApplicationClass() { try { StackTraceElement[] stackTrace = new Run ...

  10. 文件属性的生成操作build action

    文件属性的生成操作build action Action 说明 None 资源既不会被集成到程序集内,也不会打包到xap包中.不过我们可以通过设置CopyToOutputDirectory选项让其自动 ...