Executors 挖坑

Executors 挖坑

线程频繁的创建销毁是有代价的，所以Java为我们提供了线程池

线程池构造方法很多

我们一般使用Executors的工厂方法:

public static ExecutorService newFixedThreadPool(int nThreads) {
	return new ThreadPoolExecutor(nThreads, nThreads,
                                  0L, TimeUnit.MILLISECONDS,
                                  new LinkedBlockingQueue<Runnable>());
}
public static ExecutorService newCachedThreadPool() {
    return new ThreadPoolExecutor(0, Integer.MAX_VALUE,
                                  60L, TimeUnit.SECONDS,
                                  new SynchronousQueue<Runnable>());
}
public static ScheduledExecutorService newSingleThreadScheduledExecutor() {
    return new DelegatedScheduledExecutorService
        (new ScheduledThreadPoolExecutor(1));
}

除此之外Executors还为我们提供了一ForkJoin框架(一些计算密集型的应用)

我们来看一下Executors的构造方法

public ThreadPoolExecutor(int corePoolSize,//核心线程大小
                          int maximumPoolSize,//最大线程大小
                          long keepAliveTime,//超过核心线程数量的存活时间
                          TimeUnit unit,//时间单位
                          BlockingQueue<Runnable> workQueue,//阻塞队列
                          ThreadFactory threadFactory,//线程创建工厂
                          RejectedExecutionHandler handler//当线程数量超过阻塞队列容量的时候调用的处理器
                          ) {
        /*省略*/
}

我们先看一下线程池里面状态表示

    private final AtomicInteger ctl = new AtomicInteger(ctlOf(RUNNING, 0));
	//32-3==29
    private static final int COUNT_BITS = Integer.SIZE - 3;
	//2^30 - 1 相当于低29全是1
    private static final int CAPACITY   = (1 << COUNT_BITS) - 1;
    // runState is stored in the high-order bits
	// 高3位全是1
    private static final int RUNNING    = -1 << COUNT_BITS;
    private static final int SHUTDOWN   =  0 << COUNT_BITS;
    private static final int STOP       =  1 << COUNT_BITS;
    private static final int TIDYING    =  2 << COUNT_BITS;
    private static final int TERMINATED =  3 << COUNT_BITS;
    // Packing and unpacking ctl
	//获取高三位
    private static int runStateOf(int c)     { return c & ~CAPACITY; }
	//获取低29位
    private static int workerCountOf(int c)  { return c & CAPACITY; }
	//把rs低29位设置为wc
    private static int ctlOf(int rs, int wc) { return rs | wc; }

jdk实现者实际上是用一个atomicInteger来表示两个变量

就是高3位是代表一个变量，低29位是一个变量

为什么这么做呢，如果用两个变量的话，就需要多一个Atomic变量的，由于CAS，开销更大一些

//阻塞队列 可以是Linked Array Sync
private final BlockingQueue<Runnable> workQueue;
//锁
private final ReentrantLock mainLock = new ReentrantLock();
//Worker，可以理解为封装线程的一个类
private final HashSet<Worker> workers = new HashSet<Worker>();
//锁的条件对象
private final Condition termination = mainLock.newCondition();
//最大线程量
/**
* Tracks largest attained pool size. Accessed only under
* mainLock.
*/
private int largestPoolSize;
//字面意思，就是已经完成的任务数量
private long completedTaskCount;
//创建线程的工厂方法，有一个default factory
private volatile ThreadFactory threadFactory;
private volatile RejectedExecutionHandler handler;
//存活时间
private volatile long keepAliveTime;
//是否允许核心线程超时
private volatile boolean allowCoreThreadTimeOut;
//核心线程数量
private volatile int corePoolSize;
//最大线程池数量 不太理解
/**
* Maximum pool size. Note that the actual maximum is internally
* bounded by CAPACITY.
*/
private volatile int maximumPoolSize;

Worker，继承了AQS，很容易看懂

    private final class Worker
        extends AbstractQueuedSynchronizer
        implements Runnable
    {
        //序列化uid，不加这个javac会出一个警告 来自:源注释
        private static final long serialVersionUID = 6138294804551838833L;
        /** Thread this worker is running in.  Null if factory fails. */
        //封装的Thread对象
        final Thread thread;
        //第一个task
        /** Initial task to run.  Possibly null. */
        Runnable firstTask;
        /** Per-thread task counter */
        //这个worker完成的任务量
        volatile long completedTasks;
        /**
         * Creates with given first task and thread from ThreadFactory.
         * @param firstTask the first task (null if none)
         */
         //通过提交的任务创建一个worker
        Worker(Runnable firstTask) {
            setState(-1); // inhibit interrupts until runWorker
            this.firstTask = firstTask;
            //使用Thread工厂创建一个Thread 工厂内部我还没看
            this.thread = getThreadFactory().newThread(this);
        }
        /** Delegates main run loop to outer runWorker  */
        public void run() {
        //外部的方法
            runWorker(this);
        }
        // Lock methods
        //
        // The value 0 represents the unlocked state.
        // The value 1 represents the locked state.
        protected boolean isHeldExclusively() {
            return getState() != 0;
        }
        protected boolean tryAcquire(int unused) {
        //Worker is a AQS 所以这里通过CAS来判断是否能抢到这个任务
        //相当于worker内部的一个乐观锁
            if (compareAndSetState(0, 1)) {
                setExclusiveOwnerThread(Thread.currentThread());
                return true;
            }
            return false;
        }
        protected boolean tryRelease(int unused) {
            setExclusiveOwnerThread(null);
            setState(0);
            return true;
        }
        public void lock()        { acquire(1); }
        public boolean tryLock()  { return tryAcquire(1); }
        public void unlock()      { release(1); }
        public boolean isLocked() { return isHeldExclusively(); }
        //中断当前线程并且捕获中断异常
        void interruptIfStarted() {
            Thread t;
            if (getState() >= 0 && (t = thread) != null && !t.isInterrupted()) {
                try {
                    t.interrupt();
                } catch (SecurityException ignore) {
                }
            }
        }
    }

Worker实际上就是一个Thread包装类，自己还有自己的一套同步机制因为实现了AQS

我们先从execute开始看

    public void execute(Runnable command) {
        if (command == null)
            throw new NullPointerException();
        //下面三个if跟这个注释差不多
        /*
         * Proceed in 3 steps:
         *
         * 1. If fewer than corePoolSize threads are running, try to
         * start a new thread with the given command as its first
         * task.  The call to addWorker atomically checks runState and
         * workerCount, and so prevents false alarms that would add
         * threads when it shouldn't, by returning false.
         *
         * 2. If a task can be successfully queued, then we still need
         * to double-check whether we should have added a thread
         * (because existing ones died since last checking) or that
         * the pool shut down since entry into this method. So we
         * recheck state and if necessary roll back the enqueuing if
         * stopped, or start a new thread if there are none.
         *
         * 3. If we cannot queue task, then we try to add a new
         * thread.  If it fails, we know we are shut down or saturated
         * and so reject the task.
         */
        int c = ctl.get();
        //判断当前数量是不是比核心线程池数量小
        if (workerCountOf(c) < corePoolSize) {
            //尝试添加worker，添加成功直接返回就可以了
            if (addWorker(command, true))
                return;
            //获取最新的变量
            c = ctl.get();
        }
        //isRunning是判断当前线程池是不是还在运行
        //isRunning(int c) {return c < SHUTDOWN;}
        //如果线程池还在运行，那么尝试向阻塞队列提供这个任务，offer不会阻塞
        if (isRunning(c) && workQueue.offer(command)) {
            int recheck = ctl.get();
            //重新检查一下如果不运行了那么就拒绝这个任务
            if (!isRunning(recheck) && remove(command))
                reject(command);
            else if (workerCountOf(recheck) == 0)
                addWorker(null, false);
        }
        //阻塞队列无法提供，那么尝试添加到最大线程池，如果失败就拒接这个任务
        else if (!addWorker(command, false))
            reject(command);
    }

注释给的很清晰

/**
     * 检查是否可以根据当前池状态和给定的边界（核心或最大)
     * 添加新工作线程。如果是这样,工作线程数量会相应调整，如果可能的话,一个新的工作线程创建并启动
     * 将firstTask作为其运行的第一项任务。
     * 如果池已停止此方法返回false
     * 如果线程工厂在被访问时未能创建线程,也返回false
     * 如果线程创建失败，或者是由于线程工厂返回null，或者由于异常（通常是在调用Thread.start（）后的OOM）），我们干净地回滚。
     *
     * @param core if true use corePoolSize as bound, else
     * maximumPoolSize. (A boolean indicator is used here rather than a
     * value to ensure reads of fresh values after checking other pool
     * state).
     * @return true if successful
     */
private boolean addWorker(Runnable firstTask, boolean core) {
        retry:
        for (;;) {
            int c = ctl.get();
            int rs = runStateOf(c);
            // Check if queue empty only if necessary.
            // 线程池状态不是running
            //线程池关闭，而且以下三个条件满足一个就直接返回
            if (rs >= SHUTDOWN &&
                ! (rs == SHUTDOWN &&
                   firstTask == null &&
                   ! workQueue.isEmpty()))
                return false;
            for (;;) {
                int wc = workerCountOf(c);
                //线程太多了或者线程数量大于给定的边界就返回null
                if (wc >= CAPACITY ||
                    wc >= (core ? corePoolSize : maximumPoolSize))
                    return false;
                //线程数量增加了，cas增加(虽然c是负数，但是我们只关心低位)
                //如果成功了就跳转
                if (compareAndIncrementWorkerCount(c))
                    //跳出两层循环
                    break retry;
                c = ctl.get();  // Re-read ctl
                //别的线程添加成功了去外部循环重试
                if (runStateOf(c) != rs)
                    continue retry;
                // else CAS failed due to workerCount change; retry inner loop
            }
        }
        boolean workerStarted = false;
        boolean workerAdded = false;
        Worker w = null;
        try {
            //上面break到这里
            //创建一个worker
            w = new Worker(firstTask);
            final Thread t = w.thread;
            if (t != null) {
                final ReentrantLock mainLock = this.mainLock;
                mainLock.lock();
                //这里面是同步的
                //主要是因为hashset不是线程安全的
                try {
                    // Recheck while holding lock.
                    // Back out on ThreadFactory failure or if
                    // shut down before lock acquired.
                    int rs = runStateOf(ctl.get());
                    if (rs < SHUTDOWN ||
                        (rs == SHUTDOWN && firstTask == null)) {
                        //这个thread是正在运行的话说明分配错了
                        if (t.isAlive()) // precheck that t is startable
                            throw new IllegalThreadStateException();
                        //添加到set中
                        workers.add(w);
                        int s = workers.size();
                        //当前最大work的数量
                        if (s > largestPoolSize)
                            largestPoolSize = s;
                        //这行到这里说明worker添加成功了
                        workerAdded = true;
                    }
                } finally {
                    mainLock.unlock();
                }
                //检查是不是已经添加成功了
                if (workerAdded) {
                    //尝试启动，这里有可能抛出异常比如OOM
                    t.start();
                    workerStarted = true;
                }
            }
        } finally {
            //检查worker是不是已经启动了如果没启动说明出异常了或者因为线程池关闭等问题
            if (! workerStarted)
                addWorkerFailed(w);
        }
        return workerStarted;
    }

//  worker 默认执行第一个任务，然后尝试从阻塞队列里面获取任务
//  worker 线程启动后调用,while 循环(即自旋!)不断从等待队列获取任务并执行
	final void runWorker(Worker w) {
        //当前线程 worker的线程 workerthread
        Thread wt = Thread.currentThread();
        //worker的firstTask
        Runnable task = w.firstTask;
        w.firstTask = null;
        //unlock ========================= 真心不理解 但是注释是说为了允许中断
        //不过这个方法在没有锁的时候也不会抛出异常
        w.unlock(); // allow interrupts
        //
        boolean completedAbruptly = true;
        try {
            //当前有firstTask或者可以获取一个task
            while (task != null || (task = getTask()) != null) {
                //worker加锁
                w.lock();
                // If pool is stopping, ensure thread is interrupted;
                // 如果线程池已经停止，就需要确保线程安全中断
                // if not, ensure thread is not interrupted.  This
                // 如果没停止，就要确保线程池没有中断
                // requires a recheck in second case to deal with
                //不理解这个===============================================
                //这需要在第二种情况下进行重新检查，以便在关中断时处理shutdownNow竞争
                // shutdownNow race while clearing interrupt
                if ((runStateAtLeast(ctl.get(), STOP) ||
                     (Thread.interrupted() &&//当前线程被中断
                      runStateAtLeast(ctl.get(), STOP))) &&
                    !wt.isInterrupted())//当前线程被中断
                    // 试图中断当前线程
                    wt.interrupt();
                try {
                    beforeExecute(wt, task);
                    Throwable thrown = null;
                    try {
                        //这个线程执行这个
                        task.run();
                    } catch (RuntimeException x) {
                        thrown = x; throw x;
                    } catch (Error x) {
                        thrown = x; throw x;
                    } catch (Throwable x) {
                        thrown = x; throw new Error(x);
                    } finally {
                        afterExecute(task, thrown);
                    }
                } finally {
                    //任务执行完毕
                    task = null;
                    w.completedTasks++;
                    //释放锁
                    w.unlock();
                }
            }
            //这里说明所有task已经执行完毕
            completedAbruptly = false;
        } finally {
            //这里参考了别人的注释
            /*
            	1. 说明 getTask 返回 null，也就是说，这个 worker 的使命结束了，执行关闭
            	2. 任务执行过程中发生了异常
            	第一种情况，已经在代码处理了将 workCount 减 1，这个在 getTask 方法分析中说
            	第二种情况，workCount 没有进行处理，所以需要在 processWorkerExit 中处理
            */
            processWorkerExit(w, completedAbruptly);
        }
    }

暂时只看了execute的部分挖坑以后填

整理一下:

向线程池添加任务的过程:

1. 判断当前线程是不是比核心线程少 如果少就直接添加worker，如果失败的话，尝试添加到阻塞队列
2. 如果可以添加到阻塞队列，那么就尝试按照最大工作线程数量添加一个任务
3. 如果不能添加到阻塞队列，那么就尝试按照最大工作线程数量添加一个任务，如果失败了，那就拒绝这个任务

添加新的worker的时候，创建worker，用工厂给worker分配一个线程，然后添加到hashset中然后尝试执行这个worker

worker执行的时候不断自旋尝试从阻塞队列里面获取一个任务然后执行

还发现几个有趣的地方，当我们用线程池创建最大线程数量的时候是不可能到达Integer.MAX的，因为ctl是低29位记录线程数量的，当wc >= CAPACITY的时候直接创建失败

阻塞队列中的任务是通过worker消费的，worker会尝试自旋获取

为什么 newFixedThreadPool core==max 因为 LinkedBlockQueue 默认无界 永远都会添加成功所以不会执行 添加到max worker

Runnable getTask()

//getTask 根据timed判断是否使用 阻塞方法获取任务\
Runnable r = timed ?
    workQueue.poll(keepAliveTime, TimeUnit.NANOSECONDS) :
    workQueue.take();
if (r != null)
    return r;
timedOut = true;

//判断是否允许核心线程超时 或者说超过核心线程数量
boolean timed = allowCoreThreadTimeOut || wc > corePoolSize;

看一下这个参数是在哪里配置的

只查到了默认为 false 可以被set

参考注释: https://cloud.tencent.com/developer/article/1124439