C# Parallel.Invoke 实现
Parallel.Invoke应该是Parallel几个方法中最简单的一个了,我们来看看它的实现,为了方法大家理解,我尽量保留源码中的注释:
public static class Parallel
{
internal static int s_forkJoinContextID;
internal const int DEFAULT_LOOP_STRIDE = ;
internal static ParallelOptions s_defaultParallelOptions = new ParallelOptions(); public static void Invoke(params Action[] actions)
{
Invoke(s_defaultParallelOptions, actions);
}
//Executes each of the provided actions, possibly in parallel.
public static void Invoke(ParallelOptions parallelOptions, params Action[] actions)
{
if (actions == null)
{
throw new ArgumentNullException("actions");
}
if (parallelOptions == null)
{
throw new ArgumentNullException("parallelOptions");
} // Throw an ODE if we're passed a disposed CancellationToken.
if (parallelOptions.CancellationToken.CanBeCanceled && AppContextSwitches.ThrowExceptionIfDisposedCancellationTokenSource)
{
parallelOptions.CancellationToken.ThrowIfSourceDisposed();
}
// Quit early if we're already canceled -- avoid a bunch of work.
if (parallelOptions.CancellationToken.IsCancellationRequested)
throw new OperationCanceledException(parallelOptions.CancellationToken); // We must validate that the actions array contains no null elements, and also
// make a defensive copy of the actions array.
Action[] actionsCopy = new Action[actions.Length];
for (int i = ; i < actionsCopy.Length; i++)
{
actionsCopy[i] = actions[i];
if (actionsCopy[i] == null)
{
throw new ArgumentException(Environment.GetResourceString("Parallel_Invoke_ActionNull"));
}
}
// ETW event for Parallel Invoke Begin
int forkJoinContextID = ;
Task callerTask = null;
if (TplEtwProvider.Log.IsEnabled())
{
forkJoinContextID = Interlocked.Increment(ref s_forkJoinContextID);
callerTask = Task.InternalCurrent;
TplEtwProvider.Log.ParallelInvokeBegin((callerTask != null ? callerTask.m_taskScheduler.Id : TaskScheduler.Current.Id), (callerTask != null ? callerTask.Id : ),
forkJoinContextID, TplEtwProvider.ForkJoinOperationType.ParallelInvoke,
actionsCopy.Length);
}
// If we have no work to do, we are done.
if (actionsCopy.Length < ) return;
// In the algorithm below, if the number of actions is greater than this, we automatically
// use Parallel.For() to handle the actions, rather than the Task-per-Action strategy.
const int SMALL_ACTIONCOUNT_LIMIT = ;
try
{
// If we've gotten this far, it's time to process the actions.
if ((actionsCopy.Length > SMALL_ACTIONCOUNT_LIMIT) ||
(parallelOptions.MaxDegreeOfParallelism != -1 && parallelOptions.MaxDegreeOfParallelism < actionsCopy.Length))
{
// Used to hold any exceptions encountered during action processing
ConcurrentQueue<Exception> exceptionQ = null; // will be lazily initialized if necessary
// This is more efficient for a large number of actions, or for enforcing MaxDegreeOfParallelism.
try
{
// Launch a self-replicating task to handle the execution of all actions.
// The use of a self-replicating task allows us to use as many cores
// as are available, and no more. The exception to this rule is
// that, in the case of a blocked action, the ThreadPool may inject
// extra threads, which means extra tasks can run.
int actionIndex = ;
ParallelForReplicatingTask rootTask = new ParallelForReplicatingTask(parallelOptions, delegate
{
// Each for-task will pull an action at a time from the list
int myIndex = Interlocked.Increment(ref actionIndex); // = index to use + 1
while (myIndex <= actionsCopy.Length)
{
// Catch and store any exceptions. If we don't catch them, the self-replicating
// task will exit, and that may cause other SR-tasks to exit.
// And (absent cancellation) we want all actions to execute.
try
{
actionsCopy[myIndex - ]();
}
catch (Exception e)
{
LazyInitializer.EnsureInitialized<ConcurrentQueue<Exception>>(ref exceptionQ, () => { return new ConcurrentQueue<Exception>(); });
exceptionQ.Enqueue(e);
}
// Check for cancellation. If it is encountered, then exit the delegate.
if (parallelOptions.CancellationToken.IsCancellationRequested)
throw new OperationCanceledException(parallelOptions.CancellationToken);
// You're still in the game. Grab your next action index.
myIndex = Interlocked.Increment(ref actionIndex);
}
}, TaskCreationOptions.None, InternalTaskOptions.SelfReplicating);
rootTask.RunSynchronously(parallelOptions.EffectiveTaskScheduler);
rootTask.Wait();
}
catch (Exception e)
{
LazyInitializer.EnsureInitialized<ConcurrentQueue<Exception>>(ref exceptionQ, () => { return new ConcurrentQueue<Exception>(); });
// Since we're consuming all action exceptions, there are very few reasons that
// we would see an exception here. Two that come to mind:
// (1) An OCE thrown by one or more actions (AggregateException thrown)
// (2) An exception thrown from the ParallelForReplicatingTask constructor
// (regular exception thrown).
// We'll need to cover them both.
AggregateException ae = e as AggregateException;
if (ae != null)
{
// Strip off outer container of an AggregateException, because downstream
// logic needs OCEs to be at the top level.
foreach (Exception exc in ae.InnerExceptions) exceptionQ.Enqueue(exc);
}
else
{
exceptionQ.Enqueue(e);
}
}
// If we have encountered any exceptions, then throw.
if ((exceptionQ != null) && (exceptionQ.Count > ))
{
ThrowIfReducableToSingleOCE(exceptionQ, parallelOptions.CancellationToken);
throw new AggregateException(exceptionQ);
}
}
else
{
// This is more efficient for a small number of actions and no DOP support
// Initialize our array of tasks, one per action.
Task[] tasks = new Task[actionsCopy.Length];
// One more check before we begin...
if (parallelOptions.CancellationToken.IsCancellationRequested)
throw new OperationCanceledException(parallelOptions.CancellationToken);
// Launch all actions as tasks
for (int i = ; i < tasks.Length; i++)
{
tasks[i] = Task.Factory.StartNew(actionsCopy[i], parallelOptions.CancellationToken, TaskCreationOptions.None,
InternalTaskOptions.None, parallelOptions.EffectiveTaskScheduler);
}
// Optimization: Use current thread to run something before we block waiting for all tasks.
tasks[0] = new Task(actionsCopy[0]);
tasks[0].RunSynchronously(parallelOptions.EffectiveTaskScheduler); // Now wait for the tasks to complete. This will not unblock until all of
// them complete, and it will throw an exception if one or more of them also
// threw an exception. We let such exceptions go completely unhandled.
try
{
if (tasks.Length <= )
{
// for 4 or less tasks, the sequential waitall version is faster
Task.FastWaitAll(tasks);
}
else
{
// otherwise we revert to the regular WaitAll which delegates the multiple wait to the cooperative event.
Task.WaitAll(tasks);
}
}
catch (AggregateException aggExp)
{
// see if we can combine it into a single OCE. If not propagate the original exception
ThrowIfReducableToSingleOCE(aggExp.InnerExceptions, parallelOptions.CancellationToken);
throw;
}
finally
{
for (int i = ; i < tasks.Length; i++)
{
if (tasks[i].IsCompleted) tasks[i].Dispose();
}
}
}
}
finally
{
// ETW event for Parallel Invoke End
if (TplEtwProvider.Log.IsEnabled())
{
TplEtwProvider.Log.ParallelInvokeEnd((callerTask != null ? callerTask.m_taskScheduler.Id : TaskScheduler.Current.Id), (callerTask != null ? callerTask.Id : ),
forkJoinContextID);
}
}
}
}
Parallel.Invoke 的实现非常简单,如果我们Action的个数超过10或者我们制定的并行度MaxDegreeOfParallelism小于Action的个数,我们采用ParallelForReplicatingTask来完成,否则我们直接把每个Action包装成Task【Task.Factory.StartNew】。这里我们主要看看ParallelForReplicatingTask的实现。
internal class ParallelForReplicatingTask : Task
{
private int m_replicationDownCount; // downcounter to control replication internal ParallelForReplicatingTask(
ParallelOptions parallelOptions, Action action, TaskCreationOptions creationOptions, InternalTaskOptions internalOptions)
: base(action, null, Task.InternalCurrent, default(CancellationToken), creationOptions, internalOptions | InternalTaskOptions.SelfReplicating, null)
{
m_replicationDownCount = parallelOptions.EffectiveMaxConcurrencyLevel;
StackCrawlMark stackMark = StackCrawlMark.LookForMyCaller;
PossiblyCaptureContext(ref stackMark);
} internal override bool ShouldReplicate()
{
if (m_replicationDownCount == -1) return true; // "run wild"
if (m_replicationDownCount > 0) // Decrement and return true if not called with 0 downcount
{
m_replicationDownCount--;
return true;
}
return false; // We're done replicating
} internal override Task CreateReplicaTask(Action<object> taskReplicaDelegate, Object stateObject, Task parentTask, TaskScheduler taskScheduler,
TaskCreationOptions creationOptionsForReplica, InternalTaskOptions internalOptionsForReplica)
{
return new ParallelForReplicaTask(taskReplicaDelegate, stateObject, parentTask, taskScheduler, creationOptionsForReplica, internalOptionsForReplica);
}
} internal class ParallelForReplicaTask : Task
{
internal object m_stateForNextReplica;
internal object m_stateFromPreviousReplica;
internal Task m_handedOverChildReplica;
internal ParallelForReplicaTask(Action<object> taskReplicaDelegate, Object stateObject, Task parentTask, TaskScheduler taskScheduler,
TaskCreationOptions creationOptionsForReplica, InternalTaskOptions internalOptionsForReplica) :
base(taskReplicaDelegate, stateObject, parentTask, default(CancellationToken), creationOptionsForReplica, internalOptionsForReplica, taskScheduler)
{
} internal override Object SavedStateForNextReplica
{
get { return m_stateForNextReplica; } set { m_stateForNextReplica = value; }
} internal override Object SavedStateFromPreviousReplica
{
get { return m_stateFromPreviousReplica; } set { m_stateFromPreviousReplica = value; }
} internal override Task HandedOverChildReplica
{
get { return m_handedOverChildReplica; } set { m_handedOverChildReplica = value; }
}
}
ParallelForReplicatingTask的ShouldReplicate方法表示当前Task是否可以继续Replicate,每Replicate一次并行计数器减1,调用CreateReplicaTask方法创建新的ParallelForReplicaTask实例,最后调用Task的RunSynchronously方法,RunSynchronously【ExecuteSelfReplicating】才是核心实现。
public class Task : IThreadPoolWorkItem, IAsyncResult, IDisposable
{
/*
Runs the Task synchronously on the current TaskScheduler. A task may only be started and run only once. Any attempts to schedule a task a second time will result in an exception.If the target scheduler does not support running this Task on the current thread, the Task will be scheduled for execution on the scheduler, and the current thread will block until the Task has completed execution.
*/
public void RunSynchronously()
{
InternalRunSynchronously(TaskScheduler.Current, waitForCompletion: true);
}
public void RunSynchronously(TaskScheduler scheduler)
{
if (scheduler == null)
{
throw new ArgumentNullException("scheduler");
}
Contract.EndContractBlock();
InternalRunSynchronously(scheduler, waitForCompletion: true);
} internal void InnerInvokeWithArg(Task childTask)
{
InnerInvoke();
} private static void ExecuteSelfReplicating(Task root)
{
TaskCreationOptions creationOptionsForReplicas = root.CreationOptions | TaskCreationOptions.AttachedToParent;
InternalTaskOptions internalOptionsForReplicas =
InternalTaskOptions.ChildReplica | // child replica flag disables self replication for the replicas themselves.
InternalTaskOptions.SelfReplicating | // we still want to identify this as part of a self replicating group
InternalTaskOptions.QueuedByRuntime; // we queue and cancel these tasks internally, so don't allow CT registration to take place
// Important Note: The child replicas we launch from here will be attached the root replica (by virtue of the root.CreateReplicaTask call)
// because we need the root task to receive all their exceptions, and to block until all of them return
// This variable is captured in a closure and shared among all replicas.
bool replicasAreQuitting = false;
// Set up a delegate that will form the body of the root and all recursively created replicas.
Action<object> taskReplicaDelegate = null;
taskReplicaDelegate = delegate
{
Task currentTask = Task.InternalCurrent;
// Check if a child task has been handed over by a prematurely quiting replica that we might be a replacement for.
Task childTask = currentTask.HandedOverChildReplica;
if (childTask == null)
{
// Apparently we are not a replacement task. This means we need to queue up a child task for replication to progress
// Down-counts a counter in the root task.
if (!root.ShouldReplicate()) return;
// If any of the replicas have quit, we will do so ourselves.
if (Volatile.Read(ref replicasAreQuitting))
{
return;
} // Propagate a copy of the context from the root task. It may be null if flow was suppressed.
ExecutionContext creatorContext = root.CapturedContext;
childTask = root.CreateReplicaTask(taskReplicaDelegate, root.m_stateObject, root, root.ExecutingTaskScheduler,
creationOptionsForReplicas, internalOptionsForReplicas);
childTask.CapturedContext = CopyExecutionContext(creatorContext);
childTask.ScheduleAndStart(false);
}
// Finally invoke the meat of the task.
// Note that we are directly calling root.InnerInvoke() even though we are currently be in the action delegate of a child replica
// This is because the actual work was passed down in that delegate, and the action delegate of the child replica simply contains this
// replication control logic.
try
{
// passing in currentTask only so that the parallel debugger can find it
root.InnerInvokeWithArg(currentTask);
}
catch (Exception exn)
{
// Record this exception in the root task's exception list
root.HandleException(exn);
if (exn is ThreadAbortException)
{
// If this is a ThreadAbortException it will escape this catch clause, causing us to skip the regular Finish codepath
// In order not to leave the task unfinished, we now call FinishThreadAbortedTask here
currentTask.FinishThreadAbortedTask(false, true);
}
}
Object savedState = currentTask.SavedStateForNextReplica;
// check for premature exit
if (savedState != null)
{
// the replica decided to exit early
// we need to queue up a replacement, attach the saved state, and yield the thread right away Task replacementReplica = root.CreateReplicaTask(taskReplicaDelegate, root.m_stateObject, root, root.ExecutingTaskScheduler,
creationOptionsForReplicas, internalOptionsForReplicas);
// Propagate a copy of the context from the root task to the replacement task
ExecutionContext creatorContext = root.CapturedContext;
replacementReplica.CapturedContext = CopyExecutionContext(creatorContext);
replacementReplica.HandedOverChildReplica = childTask;
replacementReplica.SavedStateFromPreviousReplica = savedState;
replacementReplica.ScheduleAndStart(false);
}
else
{
// The replica finished normally, which means it can't find more work to grab.
// Time to mark replicas quitting
replicasAreQuitting = true;
// InternalCancel() could conceivably throw in the underlying scheduler's TryDequeue() method.
// If it does, then make sure that we record it.
try
{
childTask.InternalCancel(true);
}
catch (Exception e)
{
// Apparently TryDequeue threw an exception. Before propagating that exception, InternalCancel should have
// attempted an atomic state transition and a call to CancellationCleanupLogic() on this task. So we know
// the task was properly cleaned up if it was possible.
//
// Now all we need to do is to Record the exception in the root task.
root.HandleException(e);
}
// No specific action needed if the child could not be canceled
// because we attached it to the root task, which should therefore be receiving any exceptions from the child,
// and root.wait will not return before this child finishes anyway. }
};
//
// Now we execute as the root task
//
taskReplicaDelegate(null);
}
}
Task的RunSynchronously的实现路径有以下两种方式:
Task.RunSynchronously->Task.InternalRunSynchronously->TaskScheduler.TryRunInline->ThreadPoolTaskScheduler.TryExecuteTaskInline->Task.ExecuteWithThreadLocal->Task.ExecuteEntry->Task.Execute
Task.RunSynchronously->Task.InternalRunSynchronously->TaskScheduler.TryRunInline(false)->TaskScheduler.InternalQueueTask->ThreadPoolTaskScheduler.QueueTask->Task.IThreadPoolWorkItem.ExecuteWorkItem()->Task.ExecuteWithThreadLocal->Task.ExecuteEntry->Task.Execute,说白了最终会调用Task的Execute方法,在Execute方法中会检查 IsSelfReplicatingRoot是否为true【在实例ParallelForReplicatingTask时指定了参数InternalTaskOptions.SelfReplicating】,如果是则调用ExecuteSelfReplicating方法。
ExecuteSelfReplicating方法首先检查当前Task的ExecuteSelfReplicating属性是否为空【该属性也是一个Task,如果为空表示这个task运行的Action已经结束】,不为空时 我们检查Root Task是否还需要 Replicate【调用ParallelForReplicatingTask的ShouldReplicate,root.ShouldReplicate()】,然后在检查变量replicasAreQuitting是否退出循环【if (Volatile.Read(ref replicasAreQuitting)) 多线程读】,否者调用ParallelForReplicatingTask的CreateReplicaTask创建子任务,最后调用root.InnerInvokeWithArg(currentTask);,其实这里就是调用Parallel.Invoke里面的的delegate委托,每次调用只执行一个Action,currentTask.SavedStateForNextReplica这一句在Parallel.Invoke没有什么意义,但是在Parallel.For里面表示下一个要执行的Task,ParallelForReplicatingTask会执行一个Action,它可以创建子的ParallelForReplicatingTask,每个ParallelForReplicatingTask实例也会执行一个Action。实际上我没有多少个Action 就会调用多少次Task.Execute,ParallelForReplicatingTask实例个数很大程度上取决于并行度参数EffectiveMaxConcurrencyLevel,也决定ExecuteSelfReplicating调用的次数。如下线程代用流程如下: 流程1永远只调用1次【 rootTask.RunSynchronously】,流程2是ExecuteSelfReplicating代用次数,流程3是普通InnerInvoke调用次数
C# Parallel.Invoke 实现的更多相关文章
- Parallel.Invoke并行你的代码
Parallel.Invoke并行你的代码 使用Parallel.Invoke并行你的代码 优势和劣势 使用Parallel.Invoke的优势就是使用它执行很多的方法很简单,而不用担心任务或者线程的 ...
- C#并行编程中的Parallel.Invoke
一.基础知识 并行编程:并行编程是指软件开发的代码,它能在同一时间执行多个计算任务,提高执行效率和性能一种编程方式,属于多线程编程范畴.所以我们在设计过程中一般会将很多任务划分成若干个互相独立子任务, ...
- C#并行编程--命令式数据并行(Parallel.Invoke)---与匿名函数一起理解(转载整理)
命令式数据并行 Visual C# 2010和.NETFramework4.0提供了很多令人激动的新特性,这些特性是为应对多核处理器和多处理器的复杂性设计的.然而,因为他们包括了完整的新的特性,开 ...
- C#并行编程--命令式数据并行(Parallel.Invoke)
命令式数据并行 Visual C# 2010和.NETFramework4.0提供了很多令人激动的新特性,这些特性是为应对多核处理器和多处理器的复杂性设计的.然而,因为他们包括了完整的新的特性,开 ...
- Parallel.Invoke 并行的使用
Parallel类 在System.Threading.Tasks 命名空间下 下面有几个方法,这里讲一下Invoke的用法 下面我们定义几个方法方便测试 先自定义Response 防止并行的时候占 ...
- 使用Parallel.Invoke并行你的代码
优势和劣势 使用Parallel.Invoke的优势就是使用它执行很多的方法很简单,而不用担心任务或者线程的问题.然而,它并不是适合所有的场景.Parallel.Invoke有很多的劣势 如果你使用它 ...
- 看看Parallel中高度封装的三个方法,Invoke,For和ForEach
说到.net中的并行编程,也许你的第一反应就是Task,确实Task是一个非常灵活的用于并行编程的一个专用类,不可否认越灵活的东西用起来就越 复杂,高度封装的东西用起来很简单,但是缺失了灵活性,这篇我 ...
- 第九节:深究并行编程Parallel类中的三大方法 (For、ForEach、Invoke)和几大编程模型(SPM、APM、EAP、TAP)
一. 并行编程 1. 区分串行编程和串行编程 ①. 串行编程:所谓的串行编程就是单线程的作用下,按顺序执行.(典型代表for循环 下面例子从1-100按顺序执行) ②. 并行编程:充分利用多核cpu的 ...
- Parallel Programming-Parallel.Invoke
本文主要介绍Parallel.Invoke的使用. 一.使用例子 class ParallelInvoke { public void Action1() { Thread.Sleep(); Cons ...
随机推荐
- sparkStreaming序列化问题
执行sparkSTreaming+kafka 报错如下: org.apache.spark.SparkException: Task not serializable ...... Caused by ...
- redis 配置命令
Redis:是一个key/v 型数据 是nosql的一种 CAP 理论: C:多个数据节点上的数据一致: A:用户发出请求后的有限时间范围内返回结果: P:network partition,网络发 ...
- selenium_webdriver(python)控制浏览器滚动条
调过JS脚本控制 #coding=utf-8 from selenium import webdriver import time #访问百度 driver=webdriver.Firefox() d ...
- NOI2018Day2T1 屠龙勇士 set 扩展欧几里德 中国剩余定理
原文链接https://www.cnblogs.com/zhouzhendong/p/NOI2018Day2T1.html 题目传送门 - 洛谷P4774 题意 题解 首先我们仔细看一看样例可以发现如 ...
- IdentityServer4.AccessTokenValidation
IdentityServer4.AccessTokenValidation Authentication handler for ASP.NET Core 2 that allows acceptin ...
- LoadRunner的函数
一.基础函数 在VU左边导航栏中,有三个LR框架函数,分别是vuser_init(),Action(),vuser_end(). 这三个函数存在于任何Vuser类型的脚本中: ●vuser_init ...
- 认识Fiddler
一.Fiddler界面介绍.(注:下图中的功能区面板显示的是“Inspectors”的选项卡界面) 二.工具栏介绍. 1.气泡:备注.添加之后在会话栏的Comment列中显示备注内容. 2.Repla ...
- tensorflow基础架构 - 处理结构+创建一个线性回归模型+session+Variable+Placeholder
以下仅为自己的整理记录,绝大部分参考来源:莫烦Python,建议去看原博客 一.处理结构 因为TensorFlow是采用数据流图(data flow graphs)来计算, 所以首先我们得创建一个数据 ...
- day 42 mycql 数据类型
关于性别字段的存储数据类型选用 -- 性别字段 create table actor2 (id int not null auto_increment primary key,name char(32 ...
- Java实现简单计算器、抽票程序
计算器: import java.awt.BorderLayout; import java.awt.Container; import java.awt.Font; import java.awt. ...