Zookeeper笔记之基于zk的分布式配置中心

一、场景 & 需求

集群上有很多个节点运行同一个任务，这个任务会有一些可能经常改变的配置参数，要求是当配置参数改变之后能够很快地同步到每个节点上，如果将这些配置参数放在本地文件中则每次都要修改本地文件费时费力还可能会有遗漏，所以这个时候一个比较自然的想法就是将配置单独提取出来作为一个服务，比如自己开发一个http服务器提供一个接口来获取服务，这有两个问题，其一是配置下发这其实是一个推模型，当配置发生改变时需要服务器去主动推给客户端而不是客户端不断地去轮询，其二是配置中心不能是单点故障，对配置中心的可用性有一定要求，这时候如果有zookeeper集群的话直接拿来作为配置中心使用不失为一种简单的方案。

二、分析

一个配置中心的核心是什么：

1. 低延迟：配置改变后能够尽快的将最新配置同步给每一个节点。

2. 高可用：配置中心需要能够稳定不间断的提供服务。

第一点可以通过zookeeper的watcher机制实现，约定一个节点用来存放配置信息，每个客户端都监听这个节点的NodeDataChanged事件，当配置发生改变时将最新的配置更新到这个节点上（谁更新无所谓，任意一个节点都可以更新，或者做一个另外的配置管理后台用来更新都没有问题），这个节点触发NodeDataChanged事件，通知所有监听此节点NodeDataChanged事件的客户端获取此节点的最新值，因为watcher是一次性的，所以在获取最新值的时候需要重新设置监听事件，因为getData是原子性操作，所以能够保证获取到的一定是最新的值。这里需要注意的是存放在节点上的配置文件不宜过大，如果配置文件部分很大而每次变更的只是一部分的话或许可以考虑对其进行拆分，存放在多个节点上。

第二点的高可用性就是交由zookeeper集群来保证，在应用层面不需要做额外的工作。

下面是分布式配置管理中心简单的示意图：

程序运行过程中不断的重复123步骤。

三、代码实现

ConfigManager.java：

package cc11001100.zookeeper.configManager;

import cc11001100.zookeeper.utils.ZooKeeperUtil;
import org.apache.zookeeper.KeeperException;
import org.apache.zookeeper.ZooKeeper;

import java.io.IOException;

/**
 * 使用zookeeper做分布式配置管理的例子
 *
 * @author CC11001100
 */
public class ConfigManager {

	private String configNode;
	private ZooKeeper zooKeeper;

	public ConfigManager(String configNode) throws IOException {
		this.configNode = configNode;
		zooKeeper = ZooKeeperUtil.getZooKeeper();
		registerConfigChangeListener();
	}

	private void registerConfigChangeListener() {
		try {
			byte[] newConfig = zooKeeper.getData(configNode, event -> registerConfigChangeListener(), null);
			if (newConfig != null) {
				handleNewConfig(new String(newConfig, "UTF-8"));
			}
		} catch (IOException | InterruptedException | KeeperException e) {
			e.printStackTrace();
		}
	}

	protected void handleNewConfig(String newConfig) {
		System.out.println(Thread.currentThread().getName() + " config changed: " + newConfig);
	}

	public void updateConfig(String newConfig) {
		try {
			zooKeeper.setData(configNode, newConfig.getBytes(), -1);
		} catch (InterruptedException | KeeperException e) {
			e.printStackTrace();
		}
	}

}

ConfigManagerTest.java：

package cc11001100.zookeeper.configManager;

import cc11001100.zookeeper.utils.ZooKeeperUtil;
import org.apache.zookeeper.CreateMode;
import org.apache.zookeeper.KeeperException;
import org.apache.zookeeper.ZooDefs;
import org.apache.zookeeper.ZooKeeper;

import java.io.IOException;
import java.util.Random;
import java.util.concurrent.TimeUnit;

/**
 * 测试更新
 *
 * @author CC11001100
 */
public class ConfigManagerTest {

	private static void sleep(int mils) {
		try {
			TimeUnit.MILLISECONDS.sleep(mils);
		} catch (InterruptedException e) {
			e.printStackTrace();
		}
	}

	public static void main(String[] args) throws IOException, KeeperException, InterruptedException {

		final String CONFIG_NODE = "/config-node";
		ZooKeeper zooKeeper = ZooKeeperUtil.getZooKeeper();
		if (zooKeeper.exists(CONFIG_NODE, false) == null) {
			zooKeeper.create(CONFIG_NODE, "default-config".getBytes(), ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
		}

		// 模拟有10个客户端，每个都有一定几率更新配置
		for (int i = 0; i < 10; i++) {
			new Thread(() -> {
				try {
					ConfigManager configManager = new ConfigManager(CONFIG_NODE);
					Random random = new Random();
					sleep(random.nextInt(3000));
					while (true) {
						if (Math.random() < 0.2) {
							// 节点可以自己更新配置
							String newConfig = Thread.currentThread().getName() + " update, new config " + random.nextLong();
							configManager.updateConfig(newConfig);
						}
						sleep(3000);
					}
				} catch (IOException e) {
					e.printStackTrace();
				}
			}, "client-" + i).start();
		}

		// 也可以有专门的配置中心来管理更新配置
		Random random = new Random();
		while (true) {
			if (Math.random() < 0.2) {
				String newConfig = "config master update, new config " + random.nextLong();
				zooKeeper.setData(CONFIG_NODE, newConfig.getBytes(), -1);
			}
			sleep(1000);
		}

	}

}

控制台输出：

...
client-4 config changed: client-5 update, new config -845834592719987179
client-5 config changed: client-5 update, new config -845834592719987179
client-3 config changed: client-5 update, new config -845834592719987179
client-6 config changed: client-5 update, new config -845834592719987179
client-1 config changed: client-5 update, new config -845834592719987179
client-7 config changed: client-5 update, new config -845834592719987179
client-2 config changed: client-5 update, new config -845834592719987179
client-8 config changed: client-5 update, new config -845834592719987179
client-9 config changed: client-5 update, new config -845834592719987179
client-0 config changed: client-5 update, new config -845834592719987179
client-1-EventThread config changed: config master update, new config -1193927892495196391
client-2-EventThread config changed: config master update, new config -1193927892495196391
client-6-EventThread config changed: config master update, new config -1193927892495196391
client-9-EventThread config changed: config master update, new config -1193927892495196391
client-8-EventThread config changed: config master update, new config -1193927892495196391
client-3-EventThread config changed: config master update, new config -1193927892495196391
client-4-EventThread config changed: config master update, new config -1193927892495196391
client-7-EventThread config changed: config master update, new config -1193927892495196391
client-5-EventThread config changed: config master update, new config -1193927892495196391
client-0-EventThread config changed: config master update, new config -1193927892495196391
client-1-EventThread config changed: client-9 update, new config -3172102246096982581
client-6-EventThread config changed: client-9 update, new config -3172102246096982581
client-0-EventThread config changed: client-9 update, new config -3172102246096982581
client-2-EventThread config changed: client-9 update, new config -3172102246096982581
client-4-EventThread config changed: client-9 update, new config -3172102246096982581
client-7-EventThread config changed: client-9 update, new config -3172102246096982581
client-8-EventThread config changed: client-9 update, new config -3172102246096982581
client-9-EventThread config changed: client-9 update, new config -3172102246096982581
client-5-EventThread config changed: client-9 update, new config -3172102246096982581
client-3-EventThread config changed: client-9 update, new config -3172102246096982581
client-8-EventThread config changed: config master update, new config -8802002496608532059
client-2-EventThread config changed: config master update, new config -8802002496608532059
client-4-EventThread config changed: config master update, new config -8802002496608532059
client-1-EventThread config changed: config master update, new config -8802002496608532059
client-6-EventThread config changed: config master update, new config -8802002496608532059
client-7-EventThread config changed: config master update, new config -8802002496608532059
client-0-EventThread config changed: config master update, new config -8802002496608532059
client-5-EventThread config changed: config master update, new config -8802002496608532059
client-9-EventThread config changed: config master update, new config -8802002496608532059
client-3-EventThread config changed: config master update, new config -8802002496608532059
client-4-EventThread config changed: client-2 update, new config -4848584377488801943
client-1-EventThread config changed: client-2 update, new config -4848584377488801943
client-6-EventThread config changed: client-2 update, new config -4848584377488801943
client-2-EventThread config changed: client-2 update, new config -4848584377488801943
client-8-EventThread config changed: client-2 update, new config -4848584377488801943
client-7-EventThread config changed: client-2 update, new config -4848584377488801943
client-5-EventThread config changed: client-2 update, new config -4848584377488801943
client-9-EventThread config changed: client-2 update, new config -4848584377488801943
client-3-EventThread config changed: client-2 update, new config -4848584377488801943
client-0-EventThread config changed: client-2 update, new config -4848584377488801943
client-6-EventThread config changed: client-9 update, new config 6591542797374556406
client-4-EventThread config changed: client-9 update, new config 6591542797374556406
client-3-EventThread config changed: client-9 update, new config 6591542797374556406
client-9-EventThread config changed: client-9 update, new config 6591542797374556406
client-1-EventThread config changed: client-9 update, new config 6591542797374556406
client-2-EventThread config changed: client-9 update, new config 6591542797374556406
client-8-EventThread config changed: client-9 update, new config 6591542797374556406
client-7-EventThread config changed: client-9 update, new config 6591542797374556406
client-0-EventThread config changed: client-9 update, new config 6591542797374556406
client-5-EventThread config changed: client-9 update, new config 6591542797374556406
client-6-EventThread config changed: client-8 update, new config -6225415529835558983
client-4-EventThread config changed: client-8 update, new config -6225415529835558983
client-8-EventThread config changed: client-8 update, new config -6225415529835558983
client-2-EventThread config changed: client-8 update, new config -6225415529835558983
client-1-EventThread config changed: client-8 update, new config -6225415529835558983
client-5-EventThread config changed: client-8 update, new config -6225415529835558983
client-0-EventThread config changed: client-8 update, new config -6225415529835558983
client-7-EventThread config changed: client-8 update, new config -6225415529835558983
client-9-EventThread config changed: client-8 update, new config -6225415529835558983
client-3-EventThread config changed: client-8 update, new config -6225415529835558983
client-1-EventThread config changed: client-4 update, new config -703954571020619435
client-4-EventThread config changed: client-4 update, new config -703954571020619435
client-8-EventThread config changed: client-4 update, new config -703954571020619435
client-6-EventThread config changed: client-4 update, new config -703954571020619435
client-2-EventThread config changed: client-4 update, new config -703954571020619435
client-7-EventThread config changed: client-4 update, new config -703954571020619435
client-0-EventThread config changed: client-4 update, new config -703954571020619435
client-5-EventThread config changed: client-4 update, new config -703954571020619435
client-3-EventThread config changed: client-4 update, new config -703954571020619435
client-9-EventThread config changed: client-4 update, new config -703954571020619435
...

四、总结

使用zk作为配置分发的优点是低延迟、高可靠性，当然也有缺点，因为watcher是跟会话绑定的，而要维护每个会话需要一个tcp一直连接到服务器，这对集群来说也是一种负载，不过考虑到2c4g单台机器支持几百连接并发很轻松，再加上整个zookeeper集群中会有多台机器平均一下，这点负载基本忽略了。

相关资料：

1. 一篇好TM长的关于配置中心的文章 - 阿里中间件团队博客（废话有点多，但很值得一看）

.