7.4 服务远程暴露 - 创建Exporter与启动netty服务端

为了安全：服务启动的ip全部使用10.10.10.10

远程服务的暴露总体步骤：

• 将ref封装为invoker
• 将invoker转换为exporter
• 启动netty
• 注册服务到zookeeper
• 订阅
• 返回新的exporter实例

服务远程暴露的代码：

             //如果配置不是local则暴露为远程服务.(配置为local，则表示只暴露本地服务)
             if (!Constants.SCOPE_LOCAL.toString().equalsIgnoreCase(scope)) {
                 if (logger.isInfoEnabled()) {
                     logger.info("Export dubbo service " + interfaceClass.getName() + " to url " + url);
                 }
                 if (registryURLs != null && registryURLs.size() > 0
                         && url.getParameter("register", true)) {
                     for (URL registryURL : registryURLs) {
                         url = url.addParameterIfAbsent("dynamic", registryURL.getParameter("dynamic"));
                         URL monitorUrl = loadMonitor(registryURL);
                         if (monitorUrl != null) {
                             url = url.addParameterAndEncoded(Constants.MONITOR_KEY, monitorUrl.toFullString());
                         }
                         if (logger.isInfoEnabled()) {
                             logger.info("Register dubbo service " + interfaceClass.getName() + " url " + url + " to registry " + registryURL);
                         }
                         Invoker<?> invoker = proxyFactory.getInvoker(ref, (Class) interfaceClass, registryURL.addParameterAndEncoded(Constants.EXPORT_KEY, url.toFullString()));
                         Exporter<?> exporter = protocol.export(invoker);
                         exporters.add(exporter);
                     }
                 } else {
                     Invoker<?> invoker = proxyFactory.getInvoker(ref, (Class) interfaceClass, url);
                     Exporter<?> exporter = protocol.export(invoker);
                     exporters.add(exporter);
                 }
             }

首先将实现类ref封装为Invoker，之后将invoker转换为exporter，最后将exporter放入缓存List<Exporter> exporters中。

一 将实现类ref封装为Invoker

 Invoker<?> invoker = proxyFactory.getInvoker(ref, (Class) interfaceClass, registryURL.addParameterAndEncoded(Constants.EXPORT_KEY, url.toFullString()));

1  为registryURL拼接export=providerUrl参数

一开始的registryURL：

registry://10.211.55.5:2181/com.alibaba.dubbo.registry.RegistryService?application=demo-provider&client=curator&dubbo=2.0.0&pid=887&registry=zookeeper&timestamp=1507096022072

registryURL.addParameterAndEncoded(Constants.EXPORT_KEY, url.toFullString())这句代码为registryURL添加了参数并编码：（这里给出没有编码的样子）

 export=dubbo://10.10.10.10:20880/com.alibaba.dubbo.demo.DemoService?anyhost=true&application=demo-provider&dubbo=2.0.0&generic=false&interface=com.alibaba.dubbo.demo.DemoService&methods=sayHello&pid=887&side=provider&timestamp=1507096024334

2  ProxyFactory$Adaptive.getInvoker(DemoServiceImpl实例, Class<DemoService>, registryURL)

     public com.alibaba.dubbo.rpc.Invoker getInvoker(java.lang.Object arg0, java.lang.Class arg1, com.alibaba.dubbo.common.URL arg2) throws com.alibaba.dubbo.rpc.RpcException {
         if (arg2 == null)
             throw new IllegalArgumentException("url == null");
         com.alibaba.dubbo.common.URL url = arg2;
         String extName = url.getParameter("proxy", "javassist");//结果是javassist
         if(extName == null)
             throw new IllegalStateException("Fail to get extension(com.alibaba.dubbo.rpc.ProxyFactory) name from url(" + url.toString() + ") use keys([proxy])");
         com.alibaba.dubbo.rpc.ProxyFactory extension = (com.alibaba.dubbo.rpc.ProxyFactory)ExtensionLoader.getExtensionLoader(com.alibaba.dubbo.rpc.ProxyFactory.class).getExtension(extName);
         return extension.getInvoker(arg0, arg1, arg2);
     }

这里，本来是调用JavassistProxyFactory的getInvoker方法，但是JavassistProxyFactory被StubProxyFactoryWrapper给aop了。

3  StubProxyFactoryWrapper.getInvoker(DemoServiceImpl实例, Class<DemoService>, registryURL)

     public <T> Invoker<T> getInvoker(T proxy, Class<T> type, URL url) throws RpcException {
         return proxyFactory.getInvoker(proxy, type, url);
     }

4  JavassistProxyFactory.getInvoker(DemoServiceImpl实例, Class<DemoService>, registryURL)

     public <T> Invoker<T> getInvoker(T proxy, Class<T> type, URL url) {
         // TODO Wrapper类不能正确处理带$的类名
         final Wrapper wrapper = Wrapper.getWrapper(proxy.getClass().getName().indexOf('$') < 0 ? proxy.getClass() : type);
         return new AbstractProxyInvoker<T>(proxy, type, url) {
             @Override
             protected Object doInvoke(T proxy, String methodName,
                                       Class<?>[] parameterTypes,
                                       Object[] arguments) throws Throwable {
                 return wrapper.invokeMethod(proxy, methodName, parameterTypes, arguments);
             }
         };
     }

首先是创建Wrapper类：Wrapper.getWrapper(Class<DemoServiceImpl>)。该类记录了DemoServiceImpl的属性名称，方法名称等信息。关键代码如下：（完整代码见：7.2 服务本地暴露）

 import com.alibaba.dubbo.common.bytecode.Wrapper;
 import java.util.HashMap;

 public class Wrapper1 extends Wrapper {

     public static String[] pns;//property name array
     public static java.util.Map pts = new HashMap();//<property key, property value>
     public static String[] mns;//method names
     public static String[] dmns;//
     public static Class[] mts0;
     /**
      * @param o  实现类
      * @param n  方法名称
      * @param p  参数类型
      * @param v  参数名称
      * @return
      * @throws java.lang.reflect.InvocationTargetException
      */
     public Object invokeMethod(Object o, String n, Class[] p, Object[] v) throws java.lang.reflect.InvocationTargetException {
         com.alibaba.dubbo.demo.provider.DemoServiceImpl w;
         try {
             w = ((com.alibaba.dubbo.demo.provider.DemoServiceImpl) o);
         } catch (Throwable e) {
             throw new IllegalArgumentException(e);
         }
         try {
             if ("sayHello".equals(n) && p.length == 1) {
                 return ($w) w.sayHello((java.lang.String) v[0]);
             }
         } catch (Throwable e) {
             throw new java.lang.reflect.InvocationTargetException(e);
         }
         throw new com.alibaba.dubbo.common.bytecode.NoSuchMethodException("Not found method \"" + n + "\" in class com.alibaba.dubbo.demo.provider.DemoServiceImpl.");
     }
 }

创建完DemoServiceImpl的Wrapper类之后（实际上该实例在本地暴露的时候已经存入缓存了，这里只是从缓存中拿出来而已），创建一个AbstractProxyInvoker实例。

     private final T proxy;
     private final Class<T> type;
     private final URL url;

     public AbstractProxyInvoker(T proxy, Class<T> type, URL url) {
         if (proxy == null) {
             throw new IllegalArgumentException("proxy == null");
         }
         if (type == null) {
             throw new IllegalArgumentException("interface == null");
         }
         if (!type.isInstance(proxy)) {
             throw new IllegalArgumentException(proxy.getClass().getName() + " not implement interface " + type);
         }
         this.proxy = proxy;
         this.type = type;
         this.url = url;
     }

最后创建完成的AbstractProxyInvoker实例属性如下：

• proxy：DemoServiceImpl实例
• type：Class<com.alibaba.dubbo.demo.DemoService>
• url：registry://10.211.55.5:2181/com.alibaba.dubbo.registry.RegistryService?application=demo-provider&client=curator&dubbo=2.0.0&export=dubbo%3A%2F%2F10.10.10.10%3A20880%2Fcom.alibaba.dubbo.demo.DemoService%3Fanyhost%3Dtrue%26application%3Ddemo-provider%26dubbo%3D2.0.0%26generic%3Dfalse%26interface%3Dcom.alibaba.dubbo.demo.DemoService%26methods%3DsayHello%26pid%3D993%26side%3Dprovider%26timestamp%3D1507100322516&pid=993&registry=zookeeper&timestamp=1507100319830

这样我们就将ref实现类转换成了Invoker，之后在调用该invoker.invoke(Invocation invocation)的时候，会调用invoker.doInvoke(T proxy, String methodName,Class<?>[] parameterTypes, Object[] arguments)的时候，就会调用相应的实现类proxy的wrapper类的invokeMethod(proxy, methodName, parameterTypes, arguments)，该方法又会调用真实的实现类methodName方法。这里可以先给出AbstractProxyInvoker.invoke(Invocation invocation)源码：

     public Result invoke(Invocation invocation) throws RpcException {
         try {
             return new RpcResult(doInvoke(proxy, invocation.getMethodName(), invocation.getParameterTypes(), invocation.getArguments()));
         } catch (InvocationTargetException e) {
             return new RpcResult(e.getTargetException());
         } catch (Throwable e) {
             throw new RpcException("Failed to invoke remote proxy method " + invocation.getMethodName() + " to " + getUrl() + ", cause: " + e.getMessage(), e);
         }
     }

这里的proxy就是上边赋好值的proxy：DemoServiceImpl实例。而方法信息会封装在Invocation对象中，该对象在服务引用时介绍。

二  将Invoker转换为Exporter

 Exporter<?> exporter = protocol.export(invoker)

1  Protocol$Adaptive.export(com.alibaba.dubbo.rpc.Invoker AbstractProxyInvoker实例)

     public com.alibaba.dubbo.rpc.Exporter export(com.alibaba.dubbo.rpc.Invoker arg0) throws com.alibaba.dubbo.rpc.RpcException {
         if (arg0 == null)
             throw new IllegalArgumentException("com.alibaba.dubbo.rpc.Invoker argument == null");
         if (arg0.getUrl() == null)
             throw new IllegalArgumentException("com.alibaba.dubbo.rpc.Invoker argument getUrl() == null");
         com.alibaba.dubbo.common.URL url = arg0.getUrl();
         String extName = (url.getProtocol() == null ? "dubbo" : url.getProtocol());//registry
         if(extName == null)
             throw new IllegalStateException("Fail to get extension(com.alibaba.dubbo.rpc.Protocol) name from url(" + url.toString() + ") use keys([protocol])");
         com.alibaba.dubbo.rpc.Protocol extension = (com.alibaba.dubbo.rpc.Protocol)ExtensionLoader.getExtensionLoader(com.alibaba.dubbo.rpc.Protocol.class).getExtension(extName);
         return extension.export(arg0);
     }

这里，由于aop的原因，首先调用了ProtocolListenerWrapper的export(Invoker<T> invoker)，如下：

     public <T> Exporter<T> export(Invoker<T> invoker) throws RpcException {
         if (Constants.REGISTRY_PROTOCOL.equals(invoker.getUrl().getProtocol())) {
             return protocol.export(invoker);
         }
         return new ListenerExporterWrapper<T>(protocol.export(invoker),
                                               Collections.unmodifiableList(ExtensionLoader.getExtensionLoader(ExporterListener.class).getActivateExtension(invoker.getUrl(), Constants.EXPORTER_LISTENER_KEY)));
     }

由于协议是“registry”，所以不做任何处理，继续调用ProtocolFilterWrapper的export(Invoker<T> invoker)，如下：

     public <T> Exporter<T> export(Invoker<T> invoker) throws RpcException {
         if (Constants.REGISTRY_PROTOCOL.equals(invoker.getUrl().getProtocol())) {
             return protocol.export(invoker);
         }
         return protocol.export(buildInvokerChain(invoker, Constants.SERVICE_FILTER_KEY, Constants.PROVIDER));
     }

同理，由于协议是“registry”，所以不做任何处理，继续调用RegistryProtocol.export(final Invoker<T> originInvoker)，如下：

     public <T> Exporter<T> export(final Invoker<T> originInvoker) throws RpcException {
         //export invoker
         final ExporterChangeableWrapper<T> exporter = doLocalExport(originInvoker);
         //registry provider
         final Registry registry = getRegistry(originInvoker);
         final URL registedProviderUrl = getRegistedProviderUrl(originInvoker);
         registry.register(registedProviderUrl);
         // 订阅override数据
         // FIXME 提供者订阅时，会影响同一JVM即暴露服务，又引用同一服务的的场景，因为subscribed以服务名为缓存的key，导致订阅信息覆盖。
         final URL overrideSubscribeUrl = getSubscribedOverrideUrl(registedProviderUrl);
         final OverrideListener overrideSubscribeListener = new OverrideListener(overrideSubscribeUrl, originInvoker);
         overrideListeners.put(overrideSubscribeUrl, overrideSubscribeListener);
         registry.subscribe(overrideSubscribeUrl, overrideSubscribeListener);
         //保证每次export都返回一个新的exporter实例
         return new Exporter<T>() {
             public Invoker<T> getInvoker() {
                 return exporter.getInvoker();
             }

             public void unexport() {
                 try {
                     exporter.unexport();
                 } catch (Throwable t) {
                     logger.warn(t.getMessage(), t);
                 }
                 try {
                     registry.unregister(registedProviderUrl);
                 } catch (Throwable t) {
                     logger.warn(t.getMessage(), t);
                 }
                 try {
                     overrideListeners.remove(overrideSubscribeUrl);
                     registry.unsubscribe(overrideSubscribeUrl, overrideSubscribeListener);
                 } catch (Throwable t) {
                     logger.warn(t.getMessage(), t);
                 }
             }
         };
     }

该方法完成了远程暴露的全部流程。

• 将invoker转换为exporter
• 启动netty
• 注册服务到zookeeper
• 订阅
• 返回新的exporter实例

2  将invoker转换为exporter并启动netty服务

 final ExporterChangeableWrapper<T> exporter = doLocalExport(originInvoker);

doLocalExport(final Invoker<T> originInvoker)

     /**
      * 1 从invoker的URL中的Map<String, String> parameters中获取key为export的地址providerUrl，该地址将是服务注册在zk上的节点
      * 2 从 Map<String, ExporterChangeableWrapper<?>> bounds 缓存中获取key为上述providerUrl的exporter，如果有，直接返回，如果没有，创建并返回
      * @return
      */
     @SuppressWarnings("unchecked")
     private <T> ExporterChangeableWrapper<T> doLocalExport(final Invoker<T> originInvoker) {
         String key = getCacheKey(originInvoker);//根据originInvoker获取providerUrl
         ExporterChangeableWrapper<T> exporter = (ExporterChangeableWrapper<T>) bounds.get(key);
         if (exporter == null) {
             synchronized (bounds) {
                 exporter = (ExporterChangeableWrapper<T>) bounds.get(key);
                 if (exporter == null) {
                     final Invoker<?> invokerDelegete = new InvokerDelegete<T>(originInvoker, getProviderUrl(originInvoker));//存储originInvoker和providerUrl
                     exporter = new ExporterChangeableWrapper<T>((Exporter<T>) protocol.export(invokerDelegete), originInvoker);
                     bounds.put(key, exporter);
                 }
             }
         }
         return exporter;
     }

2.1 从originInvoker中获取providerUrl

该方法直接首先调用getCacheKey(final Invoker<?> originInvoker)中获取providerUrl，这里的originInvoker就是上述创建出来的AbstractProxyInvoker实例，注意他的url是registry协议的，该url的export参数的value就是我们要获取的providerUrl。获取providerUrl的源码如下：

     private String getCacheKey(final Invoker<?> originInvoker) {
         URL providerUrl = getProviderUrl(originInvoker);
         String key = providerUrl.removeParameters("dynamic", "enabled").toFullString();
         return key;
     }

     private URL getProviderUrl(final Invoker<?> origininvoker) {
         String export = origininvoker.getUrl().getParameterAndDecoded(Constants.EXPORT_KEY);
         if (export == null || export.length() == 0) {
             throw new IllegalArgumentException("The registry export url is null! registry: " + origininvoker.getUrl());
         }

         URL providerUrl = URL.valueOf(export);
         return providerUrl;
     }

之后一系列的操作，就是获取该providerUrl对应的exporter，之后放入缓存Map<String, ExporterChangeableWrapper<?>> bounds中，所以一个providerUrl只会对应一个exporter。

2.2  创建InvokerDelegete

 final Invoker<?> invokerDelegete = new InvokerDelegete<T>(originInvoker, getProviderUrl(originInvoker));

InvokerDelegete是RegistryProtocol的一个静态内部类，该类是一个originInvoker的委托类，该类存储了originInvoker，其父类InvokerWrapper还会存储providerUrl，InvokerWrapper会调用originInvoker的invoke方法，也会销毁invoker。可以管理invoker的生命周期。

     public static class InvokerDelegete<T> extends InvokerWrapper<T> {
         private final Invoker<T> invoker;

         /**
          * @param invoker
          * @param url     invoker.getUrl返回此值
          */
         public InvokerDelegete(Invoker<T> invoker, URL url) {
             super(invoker, url);
             this.invoker = invoker;
         }

         public Invoker<T> getInvoker() {
             if (invoker instanceof InvokerDelegete) {
                 return ((InvokerDelegete<T>) invoker).getInvoker();
             } else {
                 return invoker;
             }
         }
     }

InvokerWrapper的核心代码：

 public class InvokerWrapper<T> implements Invoker<T> {
     private final Invoker<T> invoker;//originInvoker
     private final URL url;//providerUrl

     public InvokerWrapper(Invoker<T> invoker, URL url) {
         this.invoker = invoker;
         this.url = url;
     }

     public boolean isAvailable() {
         return invoker.isAvailable();
     }

     public Result invoke(Invocation invocation) throws RpcException {
         return invoker.invoke(invocation);
     }

     public void destroy() {
         invoker.destroy();
     }
 }

这样一个InvokerDelegete对象就创建好了，属性如下：

• invoker：originInvoker（AbstractProxyInvoker对象）
• InvokerWrapper.invoker：originInvoker（AbstractProxyInvoker对象）
• url：providerUrl（dubbo://10.10.10.10:20880/com.alibaba.dubbo.demo.DemoService?anyhost=true&application=demo-provider&dubbo=2.0.0&generic=false&interface=com.alibaba.dubbo.demo.DemoService&methods=sayHello&pid=1035&side=provider&timestamp=1507101286063）

2.3  使用DubboProtocol将InvokerDelegete转换为Exporter

 exporter = new ExporterChangeableWrapper<T>((Exporter<T>) protocol.export(invokerDelegete), originInvoker)

2.3.1  Protocol$Adaptive.export(com.alibaba.dubbo.rpc.Invoker InvokerDelegete对象)

     public com.alibaba.dubbo.rpc.Exporter export(com.alibaba.dubbo.rpc.Invoker arg0) throws com.alibaba.dubbo.rpc.RpcException {
         if (arg0 == null)
             throw new IllegalArgumentException("com.alibaba.dubbo.rpc.Invoker argument == null");
         if (arg0.getUrl() == null)
             throw new IllegalArgumentException("com.alibaba.dubbo.rpc.Invoker argument getUrl() == null");
         com.alibaba.dubbo.common.URL url = arg0.getUrl();
         String extName = (url.getProtocol() == null ? "dubbo" : url.getProtocol());//dubbo
         if(extName == null)
             throw new IllegalStateException("Fail to get extension(com.alibaba.dubbo.rpc.Protocol) name from url(" + url.toString() + ") use keys([protocol])");
         com.alibaba.dubbo.rpc.Protocol extension = (com.alibaba.dubbo.rpc.Protocol)ExtensionLoader.getExtensionLoader(com.alibaba.dubbo.rpc.Protocol.class).getExtension(extName);
         return extension.export(arg0);
     }

该代码再贴最后一遍了。之后调用ProtocolListenerWrapper的ProtocolListenerWrapper.export(Invoker<T> InvokerDelegete)，之后调用ProtocolFilterWrapper.export(Invoker<T> InvokerDelegete)：首先对InvokerDelegete对象进行8个filter的递归包装，之后使用DubboProtocol对包装后的InvokerDelegete对象进行export。

层层包装的源码：

     /**
      * 1 根据key从url中获取相应的filter的values，再根据这个values和group去获取类上带有@Active注解的filter集合
      * 2 之后将这些filter对传入的invoker进行递归包装层invoker（就是一个链表）
      */
     private static <T> Invoker<T> buildInvokerChain(final Invoker<T> invoker, String key, String group) {
         Invoker<T> last = invoker;
         List<Filter> filters = ExtensionLoader.getExtensionLoader(Filter.class).getActivateExtension(invoker.getUrl(), key, group);
         if (filters.size() > 0) {
             for (int i = filters.size() - 1; i >= 0; i--) {
                 final Filter filter = filters.get(i);
                 final Invoker<T> next = last;
                 last = new Invoker<T>() {

                     public Class<T> getInterface() {
                         return invoker.getInterface();
                     }

                     public URL getUrl() {
                         return invoker.getUrl();
                     }

                     public boolean isAvailable() {
                         return invoker.isAvailable();
                     }

                     public Result invoke(Invocation invocation) throws RpcException {
                         return filter.invoke(next, invocation);
                     }

                     public void destroy() {
                         invoker.destroy();
                     }

                     @Override
                     public String toString() {
                         return invoker.toString();
                     }
                 };
             }
         }
         return last;
     }

上述方法中最重要的就是Invoker的Result invoke(Invocation invocation)，在该方法中，是使用了filter.invoke(next, invocation)，而这里的next又可能是另一个filter。这里我们打开一个filter来看一下源码：

 @Activate(group = Constants.PROVIDER, order = -110000)
 public class EchoFilter implements Filter {
     public Result invoke(Invoker<?> invoker, Invocation inv) throws RpcException {
         if (inv.getMethodName().equals(Constants.$ECHO) && inv.getArguments() != null && inv.getArguments().length == 1)
             return new RpcResult(inv.getArguments()[0]);
         return invoker.invoke(inv);
     }
 }

可以看到，该filter会调用传入的next的invoke方法。

这里给出被递归包装后的对象：（命名为InvokerDelegete的filter对象）

 EchoFilter
 -->ClassLoaderFilter
    -->GenericFilter
       -->ContextFilter
          -->TraceFilter
             -->TimeoutFilter
                -->MonitorFilter
                   -->ExceptionFilter
                      -->InvokerDelegete对象

2.3.2  DubboProtocol.export(Invoker<T> InvokerDelegete的filter对象)

    /**
     * 1 从invoker的url中获取将要暴露的远程服务的key：com.alibaba.dubbo.demo.DemoService:20880（实际上是：serviceGroup/serviceName:serviceVersion:port）
     * 注意：本地暴露的key就是：com.alibaba.dubbo.demo.DemoService
     * 2 打开ExchangeServer
     */
    public <T> Exporter<T> export(Invoker<T> invoker) throws RpcException {
        URL url = invoker.getUrl();
        // export service.
        String key = serviceKey(url);
        DubboExporter<T> exporter = new DubboExporter<T>(invoker, key, exporterMap);
        exporterMap.put(key, exporter);

        //export an stub service for dispaching event
        Boolean isStubSupportEvent = url.getParameter(Constants.STUB_EVENT_KEY, Constants.DEFAULT_STUB_EVENT);
        Boolean isCallbackservice = url.getParameter(Constants.IS_CALLBACK_SERVICE, false);
        if (isStubSupportEvent && !isCallbackservice) {
            String stubServiceMethods = url.getParameter(Constants.STUB_EVENT_METHODS_KEY);
            if (stubServiceMethods == null || stubServiceMethods.length() == 0) {
                if (logger.isWarnEnabled()) {
                    logger.warn(new IllegalStateException("consumer [" + url.getParameter(Constants.INTERFACE_KEY) +
                            "], has set stubproxy support event ,but no stub methods founded."));
                }
            } else {
                stubServiceMethodsMap.put(url.getServiceKey(), stubServiceMethods);
            }
        }

        openServer(url);

        return exporter;
    }

首先从“InvokerDelegete的filter对象”中的url获取key，这段代码很简单，就是获取serviceGroup/serviceName:serviceVersion:port这样形式的一个key，这里最后获取到的是com.alibaba.dubbo.demo.DemoService:20880。

之后创建DubboExporter。

2.3.2.1 DubboExporter<T>(InvokerDelegete的filter对象, "com.alibaba.dubbo.demo.DemoService:20880", exporterMap)

 public class DubboExporter<T> extends AbstractExporter<T> {
     //serviceGroup/serviceName:serviceVersion:port, 例如：com.alibaba.dubbo.demo.DemoService:20880
     private final String key;//
     //{ "com.alibaba.dubbo.demo.DemoService:20880" -> 当前的DubboExporter实例 }
     private final Map<String, Exporter<?>> exporterMap;

     public DubboExporter(Invoker<T> invoker, String key, Map<String, Exporter<?>> exporterMap) {
         super(invoker);
         this.key = key;
         this.exporterMap = exporterMap;
     }

     @Override
     public void unexport() {
         super.unexport();
         exporterMap.remove(key);
     }
 }

注意这里的exporterMap是引用传递。

父类：

 public abstract class AbstractExporter<T> implements Exporter<T> {
     protected final Logger logger = LoggerFactory.getLogger(getClass());
     private final Invoker<T> invoker;
     private volatile boolean unexported = false;

     public AbstractExporter(Invoker<T> invoker) {
         if (invoker == null)
             throw new IllegalStateException("service invoker == null");
         if (invoker.getInterface() == null)
             throw new IllegalStateException("service type == null");
         if (invoker.getUrl() == null)
             throw new IllegalStateException("service url == null");
         this.invoker = invoker;
     }

     public Invoker<T> getInvoker() {
         return invoker;
     }

     public void unexport() {
         if (unexported) {
             return;
         }
         unexported = true;
         getInvoker().destroy();
     }

     public String toString() {
         return getInvoker().toString();
     }
 }

这里，我们把一个“InvokerDelegete的filter对象”赋给了AbstractExporter的Invoker引用，也就是说从exporter中可以获取到invoker。最后在DubboProtocol.export(Invoker<T> invoker)中执行：exporterMap.put(key, exporter); 这样就将{ "com.alibaba.dubbo.demo.DemoService:20880" -> 当前的DubboExporter实例 }存储起来了。

来看一下现在的DubboExporter实例：

• key：com.alibaba.dubbo.demo.DemoService:20880
• invoker：“InvokerDelegete的filter对象”
• exporterMap：{ "com.alibaba.dubbo.demo.DemoService:20880" -> 当前的DubboExporter实例 }

2.3.2.2 开启ExchangeServer

     /**
      * 从缓存Map<String, ExchangeServer> serverMap中根据"host:port"获取ExchangeServer，如果没有，创建ExchangeServer，之后放入缓存。
      * @param url
      */
     private void openServer(URL url) {
         // find server.
         String key = url.getAddress();//10.10.10.10:20880
         //client 也可以暴露一个只有server可以调用的服务。
         boolean isServer = url.getParameter(Constants.IS_SERVER_KEY, true);
         if (isServer) {
             ExchangeServer server = serverMap.get(key);
             if (server == null) {
                 serverMap.put(key, createServer(url));
             } else {
                 //server支持reset,配合override功能使用
                 server.reset(url);
             }
         }
     }

首先从provderUrl中获取host:port作为key，之后从缓存serverMap中获取ExchangeServer，如果没有，创建ExchangeServer，最后以如下方式放入缓存：

Map<String, ExchangeServer> serverMap：{ "10.10.10.10:20880"<->ExchangeServer实例 }。

创建ExchangeServer：createServer(URL providerUrl)

     private ExchangeServer createServer(URL url) {
         //默认开启server关闭时发送readonly事件
         url = url.addParameterIfAbsent(Constants.CHANNEL_READONLYEVENT_SENT_KEY, Boolean.TRUE.toString());
         //默认开启heartbeat
         url = url.addParameterIfAbsent(Constants.HEARTBEAT_KEY, String.valueOf(Constants.DEFAULT_HEARTBEAT));
         String str = url.getParameter(Constants.SERVER_KEY, Constants.DEFAULT_REMOTING_SERVER);

         if (str != null && str.length() > 0 && !ExtensionLoader.getExtensionLoader(Transporter.class).hasExtension(str))
             throw new RpcException("Unsupported server type: " + str + ", url: " + url);

         url = url.addParameter(Constants.CODEC_KEY, Version.isCompatibleVersion() ? COMPATIBLE_CODEC_NAME : DubboCodec.NAME);
         ExchangeServer server;
         try {
             server = Exchangers.bind(url, requestHandler);
         } catch (RemotingException e) {
             throw new RpcException("Fail to start server(url: " + url + ") " + e.getMessage(), e);
         }
         str = url.getParameter(Constants.CLIENT_KEY);
         if (str != null && str.length() > 0) {
             Set<String> supportedTypes = ExtensionLoader.getExtensionLoader(Transporter.class).getSupportedExtensions();
             if (!supportedTypes.contains(str)) {
                 throw new RpcException("Unsupported client type: " + str);
             }
         }
         return server;
     }

首先是在原本providerUrl上添加参数：channel.readonly.sent=true&heartbeat=60000&codec=dubbo（其中的heartbeat参数会在HeaderExchangeServer启动心跳计时器时使用）

之后使用Exchangers.bind("添加参数后的providerUrl", requestHandler)创建ExchangeServer。首先来看一下DubboProtocol#requestHandler。这个类极其重要，后续经过层层包装后，会成为最终netty的服务端逻辑处理器。

     private ExchangeHandler requestHandler = new ExchangeHandlerAdapter() {
         public Object reply(ExchangeChannel channel, Object message) throws RemotingException {
             if (message instanceof Invocation) {
                 Invocation inv = (Invocation) message;
                 Invoker<?> invoker = getInvoker(channel, inv);
                 //如果是callback 需要处理高版本调用低版本的问题
                 if (Boolean.TRUE.toString().equals(inv.getAttachments().get(IS_CALLBACK_SERVICE_INVOKE))) {
                     String methodsStr = invoker.getUrl().getParameters().get("methods");
                     boolean hasMethod = false;
                     if (methodsStr == null || methodsStr.indexOf(",") == -1) {
                         hasMethod = inv.getMethodName().equals(methodsStr);
                     } else {
                         String[] methods = methodsStr.split(",");
                         for (String method : methods) {
                             if (inv.getMethodName().equals(method)) {
                                 hasMethod = true;
                                 break;
                             }
                         }
                     }
                     if (!hasMethod) {
                         logger.warn(new IllegalStateException("The methodName " + inv.getMethodName() + " not found in callback service interface ,invoke will be ignored. please update the api interface. url is:" + invoker.getUrl()) + " ,invocation is :" + inv);
                         return null;
                     }
                 }
                 RpcContext.getContext().setRemoteAddress(channel.getRemoteAddress());
                 return invoker.invoke(inv);
             }
             throw new RemotingException(channel, "Unsupported request: " + message == null ? null : (message.getClass().getName() + ": " + message) + ", channel: consumer: " + channel.getRemoteAddress() + " --> provider: " + channel.getLocalAddress());
         }

         @Override
         public void received(Channel channel, Object message) throws RemotingException {
             if (message instanceof Invocation) {
                 reply((ExchangeChannel) channel, message);
             } else {
                 super.received(channel, message);
             }
         }

         @Override
         public void connected(Channel channel) throws RemotingException {
             invoke(channel, Constants.ON_CONNECT_KEY);
         }

         @Override
         public void disconnected(Channel channel) throws RemotingException {
             if (logger.isInfoEnabled()) {
                 logger.info("disconected from " + channel.getRemoteAddress() + ",url:" + channel.getUrl());
             }
             invoke(channel, Constants.ON_DISCONNECT_KEY);
         }

         private void invoke(Channel channel, String methodKey) {
             Invocation invocation = createInvocation(channel, channel.getUrl(), methodKey);
             if (invocation != null) {
                 try {
                     received(channel, invocation);
                 } catch (Throwable t) {
                     logger.warn("Failed to invoke event method " + invocation.getMethodName() + "(), cause: " + t.getMessage(), t);
                 }
             }
         }

         private Invocation createInvocation(Channel channel, URL url, String methodKey) {
             String method = url.getParameter(methodKey);
             if (method == null || method.length() == 0) {
                 return null;
             }
             RpcInvocation invocation = new RpcInvocation(method, new Class<?>[0], new Object[0]);
             invocation.setAttachment(Constants.PATH_KEY, url.getPath());
             invocation.setAttachment(Constants.GROUP_KEY, url.getParameter(Constants.GROUP_KEY));
             invocation.setAttachment(Constants.INTERFACE_KEY, url.getParameter(Constants.INTERFACE_KEY));
             invocation.setAttachment(Constants.VERSION_KEY, url.getParameter(Constants.VERSION_KEY));
             if (url.getParameter(Constants.STUB_EVENT_KEY, false)) {
                 invocation.setAttachment(Constants.STUB_EVENT_KEY, Boolean.TRUE.toString());
             }
             return invocation;
         }
     };

从上可以看出在该handler中，定义了与客户端连接成功／断开连接／接受到客户端消息／相应消息，以及创造Invocation的方法。其中的getInvoker(Channel channel, Invocation inv)方法简码如下：

         String serviceKey = serviceKey(port, path, inv.getAttachments().get(Constants.VERSION_KEY), inv.getAttachments().get(Constants.GROUP_KEY));
         DubboExporter<?> exporter = (DubboExporter<?>) exporterMap.get(serviceKey);
         return exporter.getInvoker();

这不就是我们刚刚放置到exporterMap中的DubboExporter，而其中的invoker不就是我们的“filter的invokerdelegete对象”。

使用Exchangers.bind(providerUrl, ExchangeHandlerAdapter对象)创建ExchangeServer

     public static ExchangeServer bind(URL url, ExchangeHandler handler) throws RemotingException {
         if (url == null) {
             throw new IllegalArgumentException("url == null");
         }
         if (handler == null) {
             throw new IllegalArgumentException("handler == null");
         }
         url = url.addParameterIfAbsent(Constants.CODEC_KEY, "exchange");
         return getExchanger(url).bind(url, handler);
     }

     public static Exchanger getExchanger(URL url) {
         String type = url.getParameter(Constants.EXCHANGER_KEY, Constants.DEFAULT_EXCHANGER);//header
         return getExchanger(type);
     }

     public static Exchanger getExchanger(String type) {
         return ExtensionLoader.getExtensionLoader(Exchanger.class).getExtension(type);
     }

getExchanger(URL url)返回一个HeaderExchanger实例。所以ExchangeServer的创建交由HeaderExchanger来实现。

HeaderExchanger.bind(providerUrl, ExchangeHandlerAdapter对象) 

     /**
      * 1 对handler进行三次包装：首先将ExchangeHandlerAdapter赋给HeaderExchangeHandler中的ExchangeHandler handler属性；然后将创建出来的HeaderExchangeHandler赋给DecodeHandler的父类AbstractChannelHandlerDelegate的ChannelHandler handler属性
      */
     public ExchangeServer bind(URL url, ExchangeHandler handler) throws RemotingException {
         return new HeaderExchangeServer(Transporters.bind(url, new DecodeHandler(new HeaderExchangeHandler(handler))));
     }

说明：

• 这里首先对传入的ExchangeHandlerAdapter进行了两次包装，最终得到DecodeHandler实例；
• 之后，使用Transporters.bind(providerUrl, DecodeHandler对象)创建了一个NettyServer；
• 最后使用HeaderExchangeServer包装了上边的NettyServer，并启动了心跳计数器。
  • HeaderExchangeServer实例也是最终返回的ExchangeServer实例，将最终被存储在Map<String, ExchangeServer> serverMap：{ "10.10.10.10:20880"<->HeaderExchangeServer实例 }

包装ExchangeHandlerAdapter，获取DecodeHandler实例。代码比较简单，不列出来了。

最终获取到的DecodeHandler实例的层级关系：

 DecodeHandler实例
 -->HeaderExchangeHandler实例
    -->ExchangeHandlerAdapter实例

使用Transporters.bind(providerUrl, DecodeHandler对象)创建了一个NettyServer

Transporters.bind(providerUrl, DecodeHandler对象)

     public static Server bind(URL url, ChannelHandler... handlers) throws RemotingException {
         if (url == null) {
             throw new IllegalArgumentException("url == null");
         }
         if (handlers == null || handlers.length == 0) {
             throw new IllegalArgumentException("handlers == null");
         }
         ChannelHandler handler;
         if (handlers.length == 1) {
             handler = handlers[0];
         } else {
             handler = new ChannelHandlerDispatcher(handlers);
         }
         return getTransporter().bind(url, handler);
     }

     public static Transporter getTransporter() {
         return ExtensionLoader.getExtensionLoader(Transporter.class).getAdaptiveExtension();
     }

Transporter$Adaptive.bind(providerUrl, DecodeHandler对象)

     public com.alibaba.dubbo.remoting.Server bind(com.alibaba.dubbo.common.URL arg0, com.alibaba.dubbo.remoting.ChannelHandler arg1) throws com.alibaba.dubbo.remoting.RemotingException {
         if (arg0 == null)
             throw new IllegalArgumentException("url == null");
         com.alibaba.dubbo.common.URL url = arg0;
         String extName = url.getParameter("server", url.getParameter("transporter", "netty"));//netty
         if(extName == null)
             throw new IllegalStateException("Fail to get extension(com.alibaba.dubbo.remoting.Transporter) name from url(" + url.toString() + ") use keys([server, transporter])");
         com.alibaba.dubbo.remoting.Transporter extension = (com.alibaba.dubbo.remoting.Transporter)ExtensionLoader.getExtensionLoader(com.alibaba.dubbo.remoting.Transporter.class).getExtension(extName);
         return extension.bind(arg0, arg1);
     }

最后NettyServer的创建由NettyTransporter来创建。

NettyTransporter.bind(providerUrl, DecodeHandler对象)

 public class NettyTransporter implements Transporter {
     public static final String NAME = "netty";

     public Server bind(URL url, ChannelHandler listener) throws RemotingException {
         return new NettyServer(url, listener);
     }

     public Client connect(URL url, ChannelHandler listener) throws RemotingException {
         return new NettyClient(url, listener);
     }
 }

new NettyServer(providerUrl, DecodeHandler对象)

     public NettyServer(URL url, ChannelHandler handler) throws RemotingException {
         super(url, ChannelHandlers.wrap(handler, ExecutorUtil.setThreadName(url, SERVER_THREAD_POOL_NAME)));
     }

这里首先为providerUrl添加参数：threadname=DubboServerHandler-10.10.10.10:20880（ExecutorUtil.setThreadName(url, SERVER_THREAD_POOL_NAME)）；

之后，使用ChannelHandlers.wrap(DecodeHandler对象, providerUrl)对DecodeHandler对象进行了三层包装，最终得到MultiMessageHandler实例；

最后调用父类的构造器初始化NettyServer的各个属性，最后启动netty。

先看一下

ChannelHandlers.wrap(DecodeHandler对象, providerUrl)

     /**
      * 这里又是层层包裹：
      * MultiMessageHandler
      * --HeartbeatHandler
      *   --AllChannelHandler
      *     --DecodeHandler
      *       --HeaderExchangeHandler
      *         --ExchangeHandlerAdapter
      * @param handler
      * @param url
      * @return
      */
     protected ChannelHandler wrapInternal(ChannelHandler handler, URL url) {
         return new MultiMessageHandler(new HeartbeatHandler(ExtensionLoader.getExtensionLoader(Dispatcher.class)
                 .getAdaptiveExtension().dispatch(handler, url)));
     }

ExtensionLoader.getExtensionLoader(Dispatcher.class).getAdaptiveExtension()获取到一个Dispatcher$Adaptive适配类。

Dispatcher$Adaptive.dispatch(DecodeHandler对象, providerUrl)

     public com.alibaba.dubbo.remoting.ChannelHandler dispatch(com.alibaba.dubbo.remoting.ChannelHandler arg0, com.alibaba.dubbo.common.URL arg1) {
         if (arg1 == null)
             throw new IllegalArgumentException("url == null");
         com.alibaba.dubbo.common.URL url = arg1;
         String extName = url.getParameter("dispatcher", url.getParameter("dispather", url.getParameter("channel.handler", "all")));//all
         if(extName == null)
             throw new IllegalStateException("Fail to get extension(com.alibaba.dubbo.remoting.Dispatcher) name from url(" + url.toString() + ") use keys([dispatcher, dispather, channel.handler])");
         com.alibaba.dubbo.remoting.Dispatcher extension = (com.alibaba.dubbo.remoting.Dispatcher)ExtensionLoader.getExtensionLoader(com.alibaba.dubbo.remoting.Dispatcher.class).getExtension(extName);
         return extension.dispatch(arg0, arg1);
     }

这里获取到AllDispatcher，Dispatcher决定了dubbo的线程模型，指定了哪些做什么，哪些线程做什么。讲到dubbo通信的时候再写。

AllDispatcher.dispatch(DecodeHandler对象, providerUrl)

     public ChannelHandler dispatch(ChannelHandler handler, URL url) {
         return new AllChannelHandler(handler, url);
     }

new AllChannelHandler(DecodeHandler对象, providerUrl)

     public AllChannelHandler(ChannelHandler handler, URL url) {
         super(handler, url);
     }

来看其父类的WrappedChannelHandler的构造器：

WrappedChannelHandler(DecodeHandler对象, providerUrl)

     protected static final ExecutorService SHARED_EXECUTOR = Executors.newCachedThreadPool(new NamedThreadFactory("DubboSharedHandler", true));
     protected final ExecutorService executor;
     protected final ChannelHandler handler;
     protected final URL url;

     public WrappedChannelHandler(ChannelHandler handler, URL url) {
         this.handler = handler;
         this.url = url;
         executor = (ExecutorService) ExtensionLoader.getExtensionLoader(ThreadPool.class).getAdaptiveExtension().getExecutor(url);

         String componentKey = Constants.EXECUTOR_SERVICE_COMPONENT_KEY;
         if (Constants.CONSUMER_SIDE.equalsIgnoreCase(url.getParameter(Constants.SIDE_KEY))) {
             componentKey = Constants.CONSUMER_SIDE;
         }
         DataStore dataStore = ExtensionLoader.getExtensionLoader(DataStore.class).getDefaultExtension();
         dataStore.put(componentKey, Integer.toString(url.getPort()), executor);//{"java.util.concurrent.ExecutorService":{"20880":executor}}
     }

首先创建了一个共享线程池：SHARED_EXECUTOR；

之后为handler／url／executor赋值，其中executor是一个200个线程数的fixed线程池（队列为0，即同步队列）；

     public Executor getExecutor(URL url) {
         String name = url.getParameter(Constants.THREAD_NAME_KEY, Constants.DEFAULT_THREAD_NAME);//默认为dubbo，但是我们这里是DubboServerHandler-10.10.10.10:20880（就是之前设置到url上的threadname）
         int threads = url.getParameter(Constants.THREADS_KEY, Constants.DEFAULT_THREADS);//
         int queues = url.getParameter(Constants.QUEUES_KEY, Constants.DEFAULT_QUEUES);//
         return new ThreadPoolExecutor(threads, threads, 0, TimeUnit.MILLISECONDS,
                 queues == 0 ? new SynchronousQueue<Runnable>() :
                         (queues < 0 ? new LinkedBlockingQueue<Runnable>()
                                 : new LinkedBlockingQueue<Runnable>(queues)),
                 new NamedThreadFactory(name, true), new AbortPolicyWithReport(name, url));
     }

之后获取了一个数据存储器：SimpleDataStore；

最后将{"java.util.concurrent.ExecutorService":{"20880": executor}}数据存储在SimpleDataStore的ConcurrentMap<String, ConcurrentMap<String, Object>> data数据结构中。也就是说：每一个端口，有一个线程池。

注意：为什么SimpleDataSource可以做缓存来使用？

     public T getExtension(String name) {
         if (name == null || name.length() == 0)
             throw new IllegalArgumentException("Extension name == null");
         if ("true".equals(name)) {
             return getDefaultExtension();
         }
         Holder<Object> holder = cachedInstances.get(name);
         if (holder == null) {
             cachedInstances.putIfAbsent(name, new Holder<Object>());
             holder = cachedInstances.get(name);
         }
         Object instance = holder.get();
         if (instance == null) {
             synchronized (holder) {
                 instance = holder.get();
                 if (instance == null) {
                     instance = createExtension(name);
                     holder.set(instance);
                 }
             }
         }
         return (T) instance;
     }

其实，就是这样SimpleDataStore实例会存储在cachedInstances缓存中，下一次不会再创建，而是直接获取该缓存。

这样之后，一个AllChannelHandler实例就完成了，该实例属性如下：

• WrappedChannelHandler.url：dubbo://10.10.10.10:20880/com.alibaba.dubbo.demo.DemoService?anyhost=true&application=demo-provider&channel.readonly.sent=true&codec=dubbo&dubbo=2.0.0&generic=false&heartbeat=60000&interface=com.alibaba.dubbo.demo.DemoService&methods=sayHello&pid=1287&side=provider&threadname=DubboServerHandler-10.10.10.10:20880&timestamp=1507116859919
• WrappedChannelHandler.handler：DecodeHandler对象
• WrappedChannelHandler.executor：FixedThreadPool实例

当然还有一个类变量WrappedChannelHandler.SHARED_EXECUTOR=CachedThreadPool实例。

之后AllChannelHandler实例会被HeartbeatHandler进行包裹，之后HeartbeatHandler实例又会被MultiMessageHandler所包裹，最后得到的MultiMessageHandler实例的层级结构如下：

 MultiMessageHandler
 -->handler: HeartbeatHandler
    -->handler: AllChannelHandler
          -->url: providerUrl
          -->executor: FixedExecutor
          -->handler: DecodeHandler
             -->handler: HeaderExchangeHandler
                -->handler: ExchangeHandlerAdapter

MultiMessageHandler实例创建出来之后，NettyServer就开始调用其各个父类进行属性的初始化了。首先来看一下NettyServer的父类层级图：

AbstractServer：

     protected static final String SERVER_THREAD_POOL_NAME = "DubboServerHandler";
     ExecutorService executor;
     private InetSocketAddress localAddress;
     private InetSocketAddress bindAddress;
     private int accepts;
     private int idleTimeout = 600;

     public AbstractServer(URL url, ChannelHandler handler) throws RemotingException {
         super(url, handler);
         localAddress = getUrl().toInetSocketAddress();
         String host = url.getParameter(Constants.ANYHOST_KEY, false)
                 || NetUtils.isInvalidLocalHost(getUrl().getHost())
                 ? NetUtils.ANYHOST : getUrl().getHost();
         bindAddress = new InetSocketAddress(host, getUrl().getPort());
         this.accepts = url.getParameter(Constants.ACCEPTS_KEY, Constants.DEFAULT_ACCEPTS);
         this.idleTimeout = url.getParameter(Constants.IDLE_TIMEOUT_KEY, Constants.DEFAULT_IDLE_TIMEOUT);
         try {
             doOpen();
             if (logger.isInfoEnabled()) {
                 logger.info("Start " + getClass().getSimpleName() + " bind " + getBindAddress() + ", export " + getLocalAddress());
             }
         } catch (Throwable t) {
             throw new RemotingException(url.toInetSocketAddress(), null, "Failed to bind " + getClass().getSimpleName()
                     + " on " + getLocalAddress() + ", cause: " + t.getMessage(), t);
         }
         //fixme replace this with better method
         DataStore dataStore = ExtensionLoader.getExtensionLoader(DataStore.class).getDefaultExtension();
         executor = (ExecutorService) dataStore.get(Constants.EXECUTOR_SERVICE_COMPONENT_KEY, Integer.toString(url.getPort()));
     }

首先调用父类初始化属性，之后启动服务。

AbstractEndpoint：

     private Codec2 codec;
     private int timeout;
     private int connectTimeout;

     public AbstractEndpoint(URL url, ChannelHandler handler) {
         super(url, handler);
         this.codec = getChannelCodec(url);
         this.timeout = url.getPositiveParameter(Constants.TIMEOUT_KEY, Constants.DEFAULT_TIMEOUT);//
         this.connectTimeout = url.getPositiveParameter(Constants.CONNECT_TIMEOUT_KEY, Constants.DEFAULT_CONNECT_TIMEOUT);//
     }

AbstractPeer：

     private final ChannelHandler handler;
     private volatile URL url;

     public AbstractPeer(URL url, ChannelHandler handler) {
         if (url == null) {
             throw new IllegalArgumentException("url == null");
         }
         if (handler == null) {
             throw new IllegalArgumentException("handler == null");
         }
         this.url = url;
         this.handler = handler;
     }

来看一下最后初始化好的NettyServer实例：

• url：providerUrl（dubbo://10.10.10.10:20880/com.alibaba.dubbo.demo.DemoService?anyhost=true&application=demo-provider&channel.readonly.sent=true&codec=dubbo&dubbo=2.0.0&generic=false&heartbeat=60000&interface=com.alibaba.dubbo.demo.DemoService&methods=sayHello&pid=1287&side=provider&timestamp=1507116859919）
• handler：MultiMessageHandler实例
• codec：DubboCountCodec实例
• timeout：1000
• connectTimeout：3000
• idleTime：600*1000
• localAddress：10.10.10.10:20880
• bindAddress：0.0.0.0:20880
• accepts：0
• executor：null（此时的executor还没实例话，要等netty服务起来之后才会从缓存中获取之前存储在SimpleDataStore缓存中的那个200个线程数的FixedThreadPool实例）

之后，就要启动netty服务了。

     /**
      * 启动netty服务，监听客户端连接
      */
     @Override
     protected void doOpen() throws Throwable {
         NettyHelper.setNettyLoggerFactory();
         ExecutorService boss = Executors.newCachedThreadPool(new NamedThreadFactory("NettyServerBoss", true));
         ExecutorService worker = Executors.newCachedThreadPool(new NamedThreadFactory("NettyServerWorker", true));
         ChannelFactory channelFactory = new NioServerSocketChannelFactory(boss, worker, getUrl().getPositiveParameter(Constants.IO_THREADS_KEY, Constants.DEFAULT_IO_THREADS));
         bootstrap = new ServerBootstrap(channelFactory);

         final NettyHandler nettyHandler = new NettyHandler(getUrl(), this);
         channels = nettyHandler.getChannels();
         // https://issues.jboss.org/browse/NETTY-365
         // https://issues.jboss.org/browse/NETTY-379
         // final Timer timer = new HashedWheelTimer(new NamedThreadFactory("NettyIdleTimer", true));
         bootstrap.setPipelineFactory(new ChannelPipelineFactory() {
             public ChannelPipeline getPipeline() {
                 NettyCodecAdapter adapter = new NettyCodecAdapter(getCodec(), getUrl(), NettyServer.this);
                 ChannelPipeline pipeline = Channels.pipeline();
                 /*int idleTimeout = getIdleTimeout();
                 if (idleTimeout > 10000) {
                     pipeline.addLast("timer", new IdleStateHandler(timer, idleTimeout / 1000, 0, 0));
                 }*/
                 pipeline.addLast("decoder", adapter.getDecoder());
                 pipeline.addLast("encoder", adapter.getEncoder());
                 pipeline.addLast("handler", nettyHandler);
                 return pipeline;
             }
         });
         // bind
         channel = bootstrap.bind(getBindAddress());
     }

说明：

• boss线程数默认只有一个；
• worker线程数：Runtime.getRuntime().availableProcessors() + 1，为计算机核数+1；
• 服务端逻辑处理器为NettyHandler：
• 编码器为：InternalEncoder实例，内部使用NettyServer的DubboCountCodec实例来编码
• 解码器为：InternalDecoder实例，内部使用NettyServer的DubboCountCodec实例来解码

NettyHandler：

 @Sharable
 public class NettyHandler extends SimpleChannelHandler {
     private final Map<String, Channel> channels = new ConcurrentHashMap<String, Channel>(); // <ip:port, channel>
     private final URL url;
     private final ChannelHandler handler;

     public NettyHandler(URL url, ChannelHandler handler) {
         if (url == null) {
             throw new IllegalArgumentException("url == null");
         }
         if (handler == null) {
             throw new IllegalArgumentException("handler == null");
         }
         this.url = url;
         this.handler = handler;
     }

     public Map<String, Channel> getChannels() {
         return channels;
     }

     @Override
     public void channelConnected(ChannelHandlerContext ctx, ChannelStateEvent e) throws Exception {
         NettyChannel channel = NettyChannel.getOrAddChannel(ctx.getChannel(), url, handler);
         try {
             if (channel != null) {
                 channels.put(NetUtils.toAddressString((InetSocketAddress) ctx.getChannel().getRemoteAddress()), channel);
             }
             handler.connected(channel);
         } finally {
             NettyChannel.removeChannelIfDisconnected(ctx.getChannel());
         }
     }

     @Override
     public void channelDisconnected(ChannelHandlerContext ctx, ChannelStateEvent e) throws Exception {
         NettyChannel channel = NettyChannel.getOrAddChannel(ctx.getChannel(), url, handler);
         try {
             channels.remove(NetUtils.toAddressString((InetSocketAddress) ctx.getChannel().getRemoteAddress()));
             handler.disconnected(channel);
         } finally {
             NettyChannel.removeChannelIfDisconnected(ctx.getChannel());
         }
     }

     @Override
     public void messageReceived(ChannelHandlerContext ctx, MessageEvent e) throws Exception {
         NettyChannel channel = NettyChannel.getOrAddChannel(ctx.getChannel(), url, handler);
         try {
             handler.received(channel, e.getMessage());
         } finally {
             NettyChannel.removeChannelIfDisconnected(ctx.getChannel());
         }
     }

     @Override
     public void writeRequested(ChannelHandlerContext ctx, MessageEvent e) throws Exception {
         super.writeRequested(ctx, e);
         NettyChannel channel = NettyChannel.getOrAddChannel(ctx.getChannel(), url, handler);
         try {
             handler.sent(channel, e.getMessage());
         } finally {
             NettyChannel.removeChannelIfDisconnected(ctx.getChannel());
         }
     }

     @Override
     public void exceptionCaught(ChannelHandlerContext ctx, ExceptionEvent e) throws Exception {
         NettyChannel channel = NettyChannel.getOrAddChannel(ctx.getChannel(), url, handler);
         try {
             handler.caught(channel, e.getCause());
         } finally {
             NettyChannel.removeChannelIfDisconnected(ctx.getChannel());
         }
     }
 }

说明：

属性

• handler：就是当前的NettyServer实例
• url：providerUrl
• channels：存放连接到来的channel

监听连接完成／连接断开／接收到消息／发送完消息／异常捕捉事件，之后使用NettyServer实例进行相应的处理，NettyServer又会调用MultiMessageHandler实例（该handler属性位于NettyServer的父类AbstractPeer中）进行处理。

在来看编码器和解码器：

NettyCodecAdapter(DubboCountCodec实例, providerUrl, 当前的NettyServer实例)

 final class NettyCodecAdapter {
     private final ChannelHandler encoder = new InternalEncoder();
     private final ChannelHandler decoder = new InternalDecoder();
     private final Codec2 codec;
     private final URL url;
     private final int bufferSize;
     private final com.alibaba.dubbo.remoting.ChannelHandler handler;

     public NettyCodecAdapter(Codec2 codec, URL url, com.alibaba.dubbo.remoting.ChannelHandler handler) {
         this.codec = codec;
         this.url = url;
         this.handler = handler;
         int b = url.getPositiveParameter(Constants.BUFFER_KEY, Constants.DEFAULT_BUFFER_SIZE);//8*1024
         this.bufferSize = b >= Constants.MIN_BUFFER_SIZE && b <= Constants.MAX_BUFFER_SIZE ? b : Constants.DEFAULT_BUFFER_SIZE;//8*1024
     }

     public ChannelHandler getEncoder() {
         return encoder;
     }

     public ChannelHandler getDecoder() {
         return decoder;
     }

     @Sharable
     private class InternalEncoder extends OneToOneEncoder {
         @Override
         protected Object encode(ChannelHandlerContext ctx, Channel ch, Object msg) throws Exception {
             ...
             codec.encode(channel, buffer, msg);
             ...
         }
     }

     private class InternalDecoder extends SimpleChannelUpstreamHandler {
         @Override
         public void messageReceived(ChannelHandlerContext ctx, MessageEvent event) throws Exception {
            ...
             msg = codec.decode(channel, message);
            ...
         }
         ...
     }
 }

只列出核心代码：可以看到，InternalEncoder实例和InternalDecoder实例内部还是使用NettyServer的DubboCountCodec实例来编解码的。dubbo的编解码做的非常好，后续会写。

到此为止，NettyServer就创建成功了。 之后，终于执行到了：

new HeaderExchangeServer(Server NettyServer)。

     private final ScheduledExecutorService scheduled = Executors.newScheduledThreadPool(1, new NamedThreadFactory("dubbo-remoting-server-heartbeat",true));
     private final Server server;
     // 心跳定时器
     private ScheduledFuture<?> heatbeatTimer;
     // 心跳超时，毫秒。缺省0，不会执行心跳。
     private int heartbeat;
     private int heartbeatTimeout;
     private AtomicBoolean closed = new AtomicBoolean(false);

     public HeaderExchangeServer(Server server) {
         if (server == null) {
             throw new IllegalArgumentException("server == null");
         }
         this.server = server;
         this.heartbeat = server.getUrl().getParameter(Constants.HEARTBEAT_KEY, 0);//60000 在createServer(URL providerUrl)中拼接了heartbeat参数
         this.heartbeatTimeout = server.getUrl().getParameter(Constants.HEARTBEAT_TIMEOUT_KEY, heartbeat * 3);//3*60000
         if (heartbeatTimeout < heartbeat * 2) {
             throw new IllegalStateException("heartbeatTimeout < heartbeatInterval * 2");
         }
         startHeatbeatTimer();
     }

说明：

• 属性
  • scheduled：是一个有1个名字为dubbo-remoting-server-heartbeat的后台线程的定时线程池；
  • server：之前创建出来的NettyServer实例；
  • heartbeatTimer：心跳计时器
  • heartbeat：心跳时间，该参数会在HeaderExchangeServer的构造器中进行赋值，60000
  • heartbeatTimeout：心跳超时时间，超过该时间，会进行channel重连，180000
• 启动心跳计时器

startHeatbeatTimer()

     private void startHeatbeatTimer() {
         stopHeartbeatTimer();
         if (heartbeat > 0) {
             heatbeatTimer = scheduled.scheduleWithFixedDelay(
                     new HeartBeatTask(new HeartBeatTask.ChannelProvider() {
                         public Collection<Channel> getChannels() {
                             return Collections.unmodifiableCollection(HeaderExchangeServer.this.getChannels());
                         }
                     }, heartbeat, heartbeatTimeout),
                     heartbeat,
                     heartbeat,
                     TimeUnit.MILLISECONDS);
         }
     }

     private void stopHeartbeatTimer() {
         try {
             ScheduledFuture<?> timer = heatbeatTimer;
             if (timer != null && !timer.isCancelled()) {
                 timer.cancel(true);
             }
         } catch (Throwable t) {
             logger.warn(t.getMessage(), t);
         } finally {
             heatbeatTimer = null;
         }
     }

首先停掉之前的计时器，之后在线程创建开始heartbeat毫秒（60s）后执行第一次HeartBeatTask任务，之后每隔heartbeat毫秒（60s）执行一次HeartBeatTask任务。来看一下HeartBeatTask：

HeartBeatTask

 final class HeartBeatTask implements Runnable {
     private ChannelProvider channelProvider;
     private int heartbeat;//60s
     private int heartbeatTimeout;//180s

     HeartBeatTask(ChannelProvider provider, int heartbeat, int heartbeatTimeout) {
         this.channelProvider = provider;
         this.heartbeat = heartbeat;
         this.heartbeatTimeout = heartbeatTimeout;
     }

     public void run() {
         try {
             long now = System.currentTimeMillis();
             for (Channel channel : channelProvider.getChannels()) {
                 if (channel.isClosed()) {
                     continue;
                 }
                 try {
                     Long lastRead = (Long) channel.getAttribute(HeaderExchangeHandler.KEY_READ_TIMESTAMP);//"READ_TIMESTAMP"
                     Long lastWrite = (Long) channel.getAttribute(HeaderExchangeHandler.KEY_WRITE_TIMESTAMP);//"WRITE_TIMESTAMP"
                     //如果最后一次读和写在heartbeat时间（60s）内，则最后一次的读和写本身可以看作心跳；否则，需要程序发送心跳
                     if ((lastRead != null && now - lastRead > heartbeat)
                             || (lastWrite != null && now - lastWrite > heartbeat)) {
                         Request req = new Request();
                         req.setVersion("2.0.0");
                         req.setTwoWay(true);
                         req.setEvent(Request.HEARTBEAT_EVENT);
                         channel.send(req);
                         if (logger.isDebugEnabled()) {
                             logger.debug("Send heartbeat to remote channel " + channel.getRemoteAddress()
                                     + ", cause: The channel has no data-transmission exceeds a heartbeat period: " + heartbeat + "ms");
                         }
                     }
                     //如果最后一次读的时间距离现在已经超过heartbeatTimeout了，我们认为channel已经断了（因为在这个过程中，发送了三次心跳都没反应），此时channel进行重连
                     if (lastRead != null && now - lastRead > heartbeatTimeout) {
                         logger.warn("Close channel " + channel
                                 + ", because heartbeat read idle time out: " + heartbeatTimeout + "ms");
                         if (channel instanceof Client) {
                             try {
                                 ((Client) channel).reconnect();
                             } catch (Exception e) {
                                 //do nothing
                             }
                         } else {
                             channel.close();
                         }
                     }
                 } catch (Throwable t) {
                     logger.warn("Exception when heartbeat to remote channel " + channel.getRemoteAddress(), t);
                 }
             }
         } catch (Throwable t) {
             logger.warn("Unhandled exception when heartbeat, cause: " + t.getMessage(), t);
         }
     }

     interface ChannelProvider {
         Collection<Channel> getChannels();
     }
 }

说明：

• 属性
  • channelProvider在startHeatbeatTimer()中创建，并且获取了当前的HeaderExchangeServer的所有channels
  • heartbeat：60s
  • heartbeatTimeout：180s
• run()
  • 如果最后一次读和写的时间距离现在在heartbeat时间（60s）内，则最后一次的读和写本身可以看作心跳；否则，发送心跳
  • 如果最后一次读的时间距离现在已经超过heartbeatTimeout了，认为channel已经断了（因为在这个过程中，发送了三次心跳都没反应），此时channel进行重连

到现在一个完整的ExchangeServer就OK了。之后我们将创建出来的ExchangeServer实例存放在DubboProtocol的Map<String, ExchangeServer> serverMap属性中：

{ "10.10.10.10:20880" : ExchangeServer实例 }

最后，DubboProtocol.export(Invoker<T> invoker)将之前创建的DubboExporter实例返回。

2.4  创建RegistryProtocol.ExporterChangeableWrapper来封装Exporter和originInvoker

 exporter = new ExporterChangeableWrapper<T>((Exporter<T>) protocol.export(invokerDelegete), originInvoker)

     private class ExporterChangeableWrapper<T> implements Exporter<T> {
         private final Invoker<T> originInvoker;
         private Exporter<T> exporter;

         public ExporterChangeableWrapper(Exporter<T> exporter, Invoker<T> originInvoker) {
             this.exporter = exporter;
             this.originInvoker = originInvoker;
         }

         public Invoker<T> getOriginInvoker() {
             return originInvoker;
         }

         public Invoker<T> getInvoker() {
             return exporter.getInvoker();
         }

         public void setExporter(Exporter<T> exporter) {
             this.exporter = exporter;
         }

         public void unexport() {
             String key = getCacheKey(this.originInvoker);
             bounds.remove(key);
             exporter.unexport();
         }
     }

ExporterChangeableWrapper类是RegistryProtocol的私有内部类。

最后，将<providerUrl, ExporterChangeableWrapper实例>放入RegistryProtocol的属性Map<String, ExporterChangeableWrapper<?>> bounds中。

• key：dubbo://10.10.10.10:20880/com.alibaba.dubbo.demo.DemoService?anyhost=true&application=demo-provider&dubbo=2.0.0&generic=false&interface=com.alibaba.dubbo.demo.DemoService&methods=sayHello&pid=744&side=provider&timestamp=1507176748026
• value：RegistryProtocol$ExporterChangeableWrapper实例
  • originInvoker：即AbstractProxyInvoker实例属性如下：
    • proxy：DemoServiceImpl实例
    • type：Class<com.alibaba.dubbo.demo.DemoService>
    • url：registry://10.211.55.5:2181/com.alibaba.dubbo.registry.RegistryService?application=demo-provider&client=curator&dubbo=2.0.0&export=dubbo%3A%2F%2F10.10.10.10%3A20880%2Fcom.alibaba.dubbo.demo.DemoService%3Fanyhost%3Dtrue%26application%3Ddemo-provider%26dubbo%3D2.0.0%26generic%3Dfalse%26interface%3Dcom.alibaba.dubbo.demo.DemoService%26methods%3DsayHello%26pid%3D993%26side%3Dprovider%26timestamp%3D1507100322516&pid=993&registry=zookeeper&timestamp=1507100319830
  • DubboExporter实例
    • key：com.alibaba.dubbo.demo.DemoService:20880
    • invoker："InvokerDelegete的filter对象"
    • exporterMap：{ "com.alibaba.dubbo.demo.DemoService:20880" -> 当前的DubboExporter实例 }

到此为止，RegistryProtocol.export(final Invoker<T> originInvoker)的第一行代码就完成了。

     public <T> Exporter<T> export(final Invoker<T> originInvoker) throws RpcException {
         //export invoker
         final ExporterChangeableWrapper<T> exporter = doLocalExport(originInvoker);
         //registry provider
         final Registry registry = getRegistry(originInvoker);
         final URL registedProviderUrl = getRegistedProviderUrl(originInvoker);
         registry.register(registedProviderUrl);
         // 订阅override数据
         // FIXME 提供者订阅时，会影响同一JVM即暴露服务，又引用同一服务的的场景，因为subscribed以服务名为缓存的key，导致订阅信息覆盖。
         final URL overrideSubscribeUrl = getSubscribedOverrideUrl(registedProviderUrl);
         final OverrideListener overrideSubscribeListener = new OverrideListener(overrideSubscribeUrl, originInvoker);
         overrideListeners.put(overrideSubscribeUrl, overrideSubscribeListener);
         registry.subscribe(overrideSubscribeUrl, overrideSubscribeListener);
         //保证每次export都返回一个新的exporter实例
         return new Exporter<T>() {
             public Invoker<T> getInvoker() {
                 return exporter.getInvoker();
             }

             public void unexport() {
                 try {
                     exporter.unexport();
                 } catch (Throwable t) {
                     logger.warn(t.getMessage(), t);
                 }
                 try {
                     registry.unregister(registedProviderUrl);
                 } catch (Throwable t) {
                     logger.warn(t.getMessage(), t);
                 }
                 try {
                     overrideListeners.remove(overrideSubscribeUrl);
                     registry.unsubscribe(overrideSubscribeUrl, overrideSubscribeListener);
                 } catch (Throwable t) {
                     logger.warn(t.getMessage(), t);
                 }
             }
         };
     }