ccr1

Concurrency and Coordination Runtime

Jeffrey Richter

Code download available at:ConcurrentAffairs2006_09.exe(154 KB)

 Contents

The Dispatcher Class
The DispatcherQueue Class
The Port and Arbiter Classes
The Arbiter Class
Code Examples
Coordinating Several I/O Operations
Conclusion

Microsoft recently
announced the prerelease of a new Microsoft Robotics Studio for writing
applications for robots. By itself, it is interesting to see this new
toolkit, but it should have appeal even beyond just those interested in
programming robots. Under the hood, Microsoft is powering this SDK with
some very advanced technologies, including a lightweight distributed
services-oriented architecture and a common language runtime (CLR)-based
library called the Concurrency and Coordination Runtime (CCR). The CCR
makes programming asynchronous behavior much simpler than the typical
challenge of writing threaded code. This is a very significant benefit
for writing robot applications because they require handling many
processes (sensors and motors) at the same time. You should note that
this column is based on a prerelease version of both Microsoft® Robotics Studio and the CCR. All the information on these technologies is subject to change.

So
why is this interesting beyond robotics? Today many applications lack
responsiveness and scalability. It is common to see applications that
periodically hang and stop responding to user input as well as server
applications that do not respond to client requests in a timely fashion.
Who among us hasn't seen a Web browser time out due to a server not
responding quickly enough? The reason applications hang and have poor
responsiveness is almost always due to performing I/O operations (such
as file reads/writes, Web requests, and database queries) synchronously.

When
an application's thread performs synchronous I/O requests, the
application is basically giving up control of the thread's processing to
the I/O device (a hard drive, a network, or whatever). The
application's responsiveness then becomes unpredictable. Furthermore,
when threads are suspended waiting for I/O requests to complete, the
application tends to create more threads in an attempt to accomplish
more work. However, creating, scheduling, and destroying a thread
requires time and memory and can actually hurt performance rather than
improve it.

There are two main
reasons why developers tend to write code that performs synchronous I/O
instead of asynchronous I/O. The first reason is because it's easier to
write code that performs synchronous I/O. When performing asynchronous
I/O, the developer has to separate the concepts of initiating the I/O
request from the completion of the I/O request. The real problem here is
not the divorcing of the concepts as much as the syntax required. The
second reason is because it is hard to coordinate the actions you want
performed when the I/O requests complete.

The
CCR library is a managed DLL that greatly simplifies these tasks for
the programmer. The CCR offers a number of classes allowing developers a
simple object model that they can use to easily express complex
coordination patterns for dealing with completed I/O operations.
Furthermore, the CCR offers its own high-performance thread pool you can
use to execute tasks in response to completed I/O. The thread pool
offers phenomenal scalability and will maximize concurrency within your
application. When you couple the CCR with some of the new C# language
features (such as anonymous methods and iterators), you get a
developer's dream come true: an easy way to write responsive and
scalable applications.

I'll describe
the CCR's architecture and object model, and show numerous examples
that demonstrate how the CCR works as well as how to use it in your own
applications. To compile my demo code and to play with the CCR, you must
first download it. You'll find download information at the end of this
column.

To use the CCR, there are
just a few classes you'll need to become familiar with. These classes
are defined in the Microsoft.Ccr.Core namespace. Figure 1 shows the relationship between these classes. Please refer to this figure as I describe the classes.

Figure 1 CCR Class Hierarchy 

The Dispatcher Class

When
your application initializes, you'll first want to construct a
Dispatcher object that creates and manages a set of threads. In effect,
it is a thread pool. Like the CLR's thread pool, these threads call
methods (via delegates) in order to execute tasks:

 

public sealed class Dispatcher : IDisposable { public Dispatcher(); public Dispatcher(Int32 threadCount, String threadPoolName); public Dispatcher(Int32 threadCount, ThreadPriority priority, String threadPoolName); public ICollection<DispatcherQueue> DispatcherQueues { get; } ... // Other members not shown }

When you construct a Dispatcher object, you can pass to the constructor the number of threads you desire. By default, the Dispatcher creates one thread for every CPU in your computer. Notice that the number of threads created by a Dispatcher object is fixed; there is no logic in the Dispatcher for dynamically creating or destroying threads. And unlike the CLR's thread pool, there is no special thread that runs periodically checking the workload trying to predict whether threads should be dynamically added or removed from the thread pool. This streamlines the Dispatcher's thread pool logic and contributes to its high performance.

When constructing a Dispatcher, you also get to set its threads' scheduling priority. By default, the threads are created with normal priority. You can also tell the Dispatcher what string name to give the threads that it creates. Internally, when the Dispatcher creates its threads, it sets their Name property to your specified string. These names are used to help debugging and can be seen when debugging you application using the Visual Studio® debugger's Threads window.

Unlike the CLR's one-and-only thread pool, the CCR allows you to create multiple thread pools by creating multiple Dispatcher objects. This allows you to create sets of threads (at differing priorities, if desired) dedicated to processing certain kinds of tasks.

The DispatcherQueue Class

After you've constructed a Dispatcher, you'll want to construct a DispatcherQueue object. A DispatcherQueue maintains a queue of delegates that identify methods ready to execute. The Dispatcher's threads wait for entries to appear in the DispatcherQueue. Usually, the DispatcherQueue's queue is empty and therefore the Dispatcher's threads are waiting. As entries appear, Dispatcher threads wake up and execute the methods:

 

public sealed class DispatcherQueue : IDisposable { // Use CLR thread pool; not Dispatcher public DispatcherQueue(string name); public DispatcherQueue(String name, Dispatcher dispatcher); public virtual void Enqueue(ITask task); public virtual void EnqueueTimer( TimeSpan timeSpan, Port<DateTime> timerPort); public void Dispose(); ... // Other members not shown }

With the CLR's thread pool, if 1,000 items are queued up, there is no way for a new item to be processed until all of the first 1,000 items have been extracted from the thread pool's queue. But with the CCR you can use one DispatcherQueue object for most work items and use another DispatcherQueue object for high-priority work items. The Dispatcher's threads dequeue entries from all the DispatcherQueue objects associated with it in a round-robin fashion. I should also point out that calling the constructor that doesn't take a Dispatcher argument makes it possible to create a DispatcherQueue object that queues its tasks to the CLR's thread pool instead of using a Dispatcher's thread pool.

It is common for an application to construct its Dispatcher and DispatcherQueue objects during initialization and use them throughout the remainder of the application's lifetime. So, at this point, we'll turn our attention to the classes that an application tends to construct, use for a short time, and then discard.

The Port and Arbiter Classes

A generic Port<T> object represents a queue of items, all of type T. You can think of a Port as a way to queue an input argument to a callback method, which is analogous to the state argument that you pass to ThreadPool's QueueUserWorkItem method.

When asynchronous I/O operations complete, their result will be posted to a Port object. I'll explain how this happens later in this column. A Port object also has zero or more ReceiverTask objects associated with it. When an item gets posted, the ReceiverTask objects determine how to coordinate and process the item. In the Port<T> class definition that follows I do not show the methods that register and unregister ReceiverTask objects because you will not usually call these methods yourself. Instead, you will use the Arbiter's methods to create ReceiverTask objects to register these objects with a Port object:

 

public class Port<T> : /* interfaces not shown */ { public Port(); public virtual void Post(T item); // Other members not shown }

The Arbiter Class

The Arbiter class, shown in Figure 2, is what you use to tap into the CCR's coordination features; you will use this class a lot when programming against the CCR. Arbiter is a static class that defines a bunch of methods that are factories for creating other objects. Specifically, when you call one of Arbiter's static factory methods, the method constructs an arbiter that has fields that refer to one or more ReceiverTask objects that are also constructed.

 Figure 2 The Arbiter Class

 

public static class Arbiter { public static ITask FromHandler(Handler handler); public static ITask FromIteratorHandler(Handler handler); public static Receiver<T> Receive<T>( Boolean persist, Port<T> port, Handler<T> handler); public static JoinSinglePortReceiver MultipleItemReceive<T>( Boolean persist, Port<T> port, Int32 itemCount, VariableArgumentHandler<T> handler); public static JoinReceiver MultiplePortReceive<T>( Boolean persist, Port<T>[] ports, VariableArgumentHandler<T> handler); public static JoinReceiver JoinedReceive<T0, T1>( Boolean persist, Port<T0> port0, Port<T1> port1, Handler<T0, T1> handler); public static Choice Choice(params ReceiverTask[] receivers); public static Choice Choice<T0, T1>(PortSet<T0, T1> resultPort, Handler<T0> handler0, Handler<T1> handler1); public static Interleave Interleave( TeardownReceiverGroup teardown, ExclusiveReceiverGroup exclusive, ConcurrentReceiverGroup concurrent); public static void Activate( DispatcherQueue dispatcherQueue, ITask arbiter); ... // Other members not shown }

Figure 3 shows the common arbiters offered by the CCR with a brief description indicating what each arbiter does. Note that the CCR defines even more arbiters for less-common scenarios. For example, there is a JoinReceiver class that calls a method when many Port objects (of varying data types) have items posted into them. Also note that you can define your own arbiters, but this is a very advanced feature which I won't describe here. If you are interested in understanding arbiters more, I encourage you to examine the types in the Microsoft.Ccr.Core and Microsoft.Ccr.Core.Arbiters namespaces.

 Figure 3 Common CCR Arbiters In Order From Simple To Complex

Arbiter	Description
FromHandler	Indicates a method that should be queued to a DispatcherQueue. The method will just execute. This simple arbiter is not associated with any Port object.
FromIteratorHandler	Indicates an enumerator method that should be queued to a DispatcherQueue. Each item enumerated identifies a simple operation that should execute. This simple arbiter is not associated with any Port object.
Receive	Indicates a method that should be called to process a single item from single port. The Handler delegate takes one parameter (of the Port’s item type).
MultipleItemReceive	Indicates a method that should be called to process multiple items from a single Port object. The VariableArgumentHandler delegate takes one parameter (an array of the Port’s item type).
MultiplePortReceive	Indicates a method that should be called to process a single item from multiple Port objects (one item per Port object; items must be of the same type). The VariableArgumentHandler delegate takes one parameter (an array of the Port’s item type).
JoinedReceive	Indicates a method that should be called to process multiple items from two Port objects (one item per Port object; items can be of different types). The Handler delegate takes two parameters (of the Ports’ item types).
Choice	Indicates a set of methods of which only one should execute. The CCR ensures that one and only one of the specified methods execute. Arbiters passed to Choice must be non-persistent arbiters.
Interleave	Indicates a set of methods that should be called back to process items from various Port objects. An arbiter can be part of a TeardownReceiverGroup, an ExclusiveReceiverGroup, or a ConcurrentReceiverGroup. This arbiter is similar to a reader/writer thread synchronization lock.
The CCR ensures that a TeardownReceiverGroup arbiter will not execute concurrently with an arbiter from either the ExclusiveReceiverGroup or the ConcurrentReceiverGroup. In addition, the CCR ensures that no other arbiter will execute after a TeardownReceiverGroup arbiter executes. Arbiters that are part of the TeardownReceiverGroup must be non-persistent arbiters.
The CCR ensures that an ExclusiveReceiverGroup arbiter will not execute concurrently with an arbiter from either the Tear­downReceiverGroup or the ConcurrentReceiverGroup. An ExclusiveReceiverGroup arbiter is identical to a TeardownReceiver­Group arbiter except that, once executed, other arbiters (including itself) may execute again in the future.
The CCR allows a ConcurrentReceiverGroup arbiter to execute concurrently with other ConcurrentReceiverGroup arbiters. However, the CCR ensures that a ConcurrentReceiverGroup arbiter will not run concurrently with a TeardownReceiverGroup or ExclusiveReceiverGroup arbiter. Once executed, a Con­currentReceiverGroup arbiter will execute again in the future.

You'll notice that some arbiters (Receive, MultipleItemReceive, MultiplePortReceive, and JoinedReceive) take a Boolean value indicating whether the arbiter is persistent. Also, some arbiters (such as Choice and Interleave's TeardownReceiverGroup) require that their arbiters always be non-persistent. Understanding an arbiter's persistence is very important to working effectively with the CCR and I will explain persistence as I walk through some code examples.

After you've composed all the arbiters that describe your desired coordination, you must activate the arbiters by calling Arbiter's Activate method. Calling the Active method is very important because it registers all of the arbiter's ReceiverTask objects with the Ports and it tells the arbiter which DispatcherQueue to post work item tasks to as Port items are processed. If you forget to activate your arbiters, items will continue to queue up in Port objects but they will never get processed!

As items are posted into a Port object, the registered ReceiverTask objects send the items to their arbiter object which decides what callback method should ultimately execute to process the posted item. The arbiter object then queues the work item task into a DispatcherQueue, and a Dispatcher (or CLR thread pool) thread will ultimately execute the method processing the posted item.

Code Examples

The best way to understand the CCR and how it works is to examine several code examples while I explain what is happening. To this end, I have created a program with a bunch of small methods in it. Each method demonstrates some part of the CCR. I will now walk through these methods, all of which can be found in the CCRDemos.cs file in the download for this column available from the MSDN®Magazine Web site.

Initialization and Shutdown The Main method of my example demonstrates a common way to initialize the CCR. It simply creates a Dispatcher and a DispatcherQueue and then it executes various methods that demonstrate uses of the CCR. The DispatcherQueue object is passed to each demo method so that all methods use the same work item queue and thread pool. Since the Dispatcher object is constructed in a using statement, it will have its Dispose method called just before Main returns, exiting the application. The Dispatcher's Dispose method tells all the thread pool threads to gracefully exit, and after they have all exited, the Dispose method returns to its caller (Main in Figure 4). The demos make use of some helper methods as well:

 

private static void Msg(String format, params Object[] args) { Console.Write("ThreadID={0}: ", Thread.CurrentThread.ManagedThreadId); Console.WriteLine(format, args); } private static void HitEnter() { Thread.Sleep(250); // Give an operation a chance to complete Console.Write("Hit <Enter> to continue demo"); Console.ReadLine(); Console.WriteLine(); }

 Figure 4 Initialization and Shutdown in Main

 

public static void Main() { // Creates a Dispatcher (thread pool) using (Dispatcher dispatcher = new Dispatcher(0, "CCR Demo Threads")) { // Create a DispatcherQueue that queues work // item tasks to the Dispatcher DispatcherQueue dq = new DispatcherQueue( "CCR Demo DispatcherQueue", dispatcher); // Execute the various demos SpawnDemo(dq); PortArbiterDemo(dq); MultipleItemDemo(dq); ... } }

SpawnDemo The SpawnDemo method demonstrates how to spawn a task using the CCR's thread pool. In my example, I use the C# anonymous method feature to define a callback method that calls my Msg method. C# will automatically construct a delegate over this anonymous method and that delegate is passed to Arbiter's static FromHandler method which wraps the delegate into another object used natively by the CCR. Then Arbiter's static Activate method is called to queue the callback method to the specified DispatcherQueue object. This simple task spawning is similar to using ThreadPool.QueueUserWorkItem:

 

private static void SpawnDemo(DispatcherQueue dq) { Arbiter.Activate(dq, Arbiter.FromHandler( delegate { Msg("Handler"); })); HitEnter(); // wait for user input to continue }

PortArbiterDemo The PortArbiterDemo method demonstrates how Port and Arbiter objects work together. In this method, I first construct a Port<String> object which internally contains a first-in-first-out (FIFO) queue capable of holding references to String objects. A Port object also maintains a list of registered ReceiverTask arbiter objects. Then, I call the Post method to have the Port process a String item ("StringA"):

 

private static void PortArbiterDemo(DispatcherQueue dq) { Port<String> stringPort = new Port<String>(); stringPort.Post("StringA");

When an item is posted to a port, the Post method traverses the list of ReceiverTask objects to see if any of them want to handle the item. If no ReceiverTask objects are registered or if none of the registered ReceiverTask objects want the item, the item is added to the Port's internal queue. In my example, I have not yet registered any ReceiverTask objects and therefore "StringA" is placed in the Port's queue.

The next thing the PortArbiterDemo method does is call Arbiter's Receive method which creates a ReceiverTask object. The first argument, false, indicates that this ReceiverTask object should not be persistent. In other words, once this ReceiverTask object processes an item, it should not be used to process another item. The second argument, stringPort, tells the ReceiverTask object which Port object it should be watching for items. The third argument, an anonymous method delegate, tells the ReceiverTask object what method should be called to process the Port's item.

 

Arbiter.Activate(dq, Arbiter.Receive(false, stringPort, delegate(String s) { Msg("Handling string={0}", s); }));

It is important to note that calling Arbiter's Receive method simply constructs a ReceiverTask object. This object is not registered or activated with a Port object yet. Furthermore, there needs to be a way to tell the ReceiverTask object which DispatcherQueue object to queue the delegate callback method to. The call to Arbiter's Activate method does both of these things.

When Arbiter's Activate method is called, it registers the ReceiverTask object with the Port. During the registration process, the Port's previously queued items are scanned. If the ReceiverTask object wants the item, it will process it by queuing the callback delegate into the DispatcherQueue so that a Dispatcher thread will execute the code that processes the item.

In my example, I posted an item to the Port and then registered a ReceiverTask object. However, it is possible to register ReceiverTask objects with a Port first and then post items to the Port. In this case, the items are processed as they are posted and are only placed in the queue if no ReceiverTask object wants to process it. This is very efficient.

PostArbiterDemo then posts "StringB" to the port:

 

stringPort.Post("StringB");

Since I created my ReceiverTask object by passing false for the persistent argument, it is allowed to process just one Port item and then the ReceiverTask object is automatically unregistered from the Port. When the PortArbiterDemo method posts "StringB", there are no registered ReceiverTask objects and "StringB" gets placed in the Port's queue; the string is not displayed in the console window.

Next, I create another ReceiverTask object but this time, I pass true for the persistent argument. Now, when this ReceiverTask object is registered with the port, it will not automatically unregister itself and therefore it will be used to process all items currently in the Port's queue as well as new items that get posted to the Port. So, after activating this new ReceiverTask object, "StringB" appears in the console window, and all the items posted to the Port via the for loop also appear in the console window.

 

Arbiter.Activate(dq, Arbiter.Receive(true, stringPort, delegate(String s) { Msg("Handling string={0}", s); })); for (int i = 0; i < 10; i++) { stringPort.Post("String #" + i.ToString()); }

MultipleItemDemo The MultipleItemDemo method demonstrates how to process Port items in batches. In this method, I first construct a Port<String> object. Then I call Arbiter's MultipleItemReceive method to create a ReceiverTask object that knows how to process multiple items. The MultipleItemReceive method is similar to the Receive method except you also pass to it the number of items that must be posted to the Port object before processing any of them and the callback delegate must identify a method that takes an array of items instead of a single item:

 

private static void MultipleItemDemo(DispatcherQueue dq) { Port<String> stringPort = new Port<String>(); Arbiter.Activate(dq, Arbiter.MultipleItemReceive(true, stringPort, 10, delegate(String[] strings) { Msg("Ten strings={0}", String.Join(", ", strings)); })); for (int i = 0; i < 50; i++) stringPort.Post(i.ToString()); HitEnter(); }

In my example, I passed 10 to the MultipleItemReceive method and so the delegate will get called once for every 10 items posted to the Port. In this example, for variety, I activated the ReceiverTask object with the Port before I posted any items into it. I then have a for loop that posts 50 items, causing the callback to execute five times. When I run this demo, I get the following output:

 

ThreadID=12: Ten strings=0, 1, 2, 3, 4, 5, 6, 7, 8, 9 ThreadID=12: Ten strings=10, 11, 12, 13, 14, 15, 16, 17, 18, 19 ThreadID=12: Ten strings=20, 21, 22, 23, 24, 25, 26, 27, 28, 29 ThreadID=12: Ten strings=30, 31, 32, 33, 34, 35, 36, 37, 38, 39 ThreadID=12: Ten strings=40, 41, 42, 43, 44, 45, 46, 47, 48, 49 Hit <Enter> to continue demo

At this point, you should have a pretty good sense of how the pieces fit together. You should also have a sense of the kinds of coordination the CCR allows you to pull off. The examples so far have been pretty basic and all of them have been using just one Port object at a time. When you really start working with CCR, you'll see that you can get quite sophisticated and you'll frequently be coordinating items from multiple Port objects together.

AsyncStreamDemo The AsyncStreamDemo method, shown in Figure 5, demonstrates how to read data from a file asynchronously. In this method, I first construct a FileStream object passing a file that I intend to read data from and the FileOptions.Asynchronous flag. The FileStream class already offers methods to perform asynchronous I/O: BeginRead/EndRead and BeginWrite/EndWrite. I have written some adapter methods that can map or translate methods that use the CLR's Asynchronous Programming Model (APM) into the CCR's programming model. My adapter methods are all defined in ApmToCcrAdapters that accompanies the source code for this column.

 Figure 5 AsyncStreamDemo

 

private static void AsyncStreamDemo(DispatcherQueue dq) { FileStream fs = new FileStream(@"C:\Boot.ini", FileMode.Open, FileAccess.Read, FileShare.ReadWrite, 8 * 1024, FileOptions.Asynchronous); Byte[] data = new Byte[10000]; Port<Int32> bytesReadPort = null; Port<Exception> failurePort = null; ApmToCcrAdapters.Read(fs, data, 0, data.Length, ref bytesReadPort, ref failurePort); Arbiter.Activate(dq, Arbiter.Choice( Arbiter.Receive(false, bytesReadPort, delegate(Int32 bytesRead) { Array.Resize(ref data, bytesRead); Msg("Read completed, bytes read={0}, data follows:{1}{2}", data.Length, Environment.NewLine, Encoding.ASCII.GetString(data)); }), Arbiter.Receive(false, failurePort, delegate(Exception e) { Msg("Read failed, error={0}", e.Message); }))); HitEnter(); }

Basically, when you initiate an asynchronous Read operation on a Stream object, there are two possible results: the I/O can complete returning the number of bytes read (as an Int32) or the operation can fail, indicated with an Exception. So, in my AsyncStreamDemo method, I create a Port<Int32> variable (called bytesReadPort) and a Port<Exception> variable (called failurePort). Both variables are initialized to null.

Next I call my ApmToCcrAdapters type's static Read method. This Read method takes a reference to any Stream-derived object as its first argument. I pass in the arguments that you would normally pass to Stream's Read method: buffer, offset, and count. Then I pass in the reference to the two Port objects. My ApmToCcrAdapters type's static Read method internally constructs the two Port objects and then it starts the asynchronous I/O operation by calling the Stream's BeginRead method.

When the I/O operation completes, an internal callback method (provided by my adapter code) is invoked which checks the result. If the I/O completes successfully, then the number of bytes read (returned from Stream's EndRead method) is posted into the Port<Int32> object. If the I/O completes unsuccessfully, the internal method calls EndRead which throws an exception. The internal method catches this exception and posts the Exception-derived object it into the Port<Exception> object. So, for any single I/O operation only one of the two Port objects will have anything posted into it.

Now I need to tell the CCR how to coordinate the possible results. To do this, I call Arbiter's Choice method, which tells the CCR that only one of the ReceiverTask objects passed to it should be executed. In my example, the first call to Arbiter's Receive method tells the CCR what to execute should the asynchronous I/O operation complete successfully—its anonymous method displays the number of bytes read and the data read from the stream as well. The second call to Arbiter's Receive method tells the CCR what to execute should the asynchronous I/O operation complete unsuccessfully—the Exception object's message is displayed.

Note that when using Arbiter's Choice method, all of the ReceiverTask objects you pass to it must have false passed for their persistent argument. That is, Choice is designed to select one of the specified ReceiverTask objects and then all the ReceiverTask objects are effectively unregistered from their respective Port objects. This is how the CCR ensures that only one ReceiverTask object is chosen (which is what Choice is all about).

AsyncStreamPortSetDemo The AsyncStreamPortSetDemo method demonstrates how to accomplish the exact same job as shown in the AsyncStreamDemo using slightly less code. Because it is so common to use two Port objects together—one for success and the other for failure—the Microsoft.Ccr.Core namespace defines a PortSet<T0, T1> class:

 

PortSet<Int32, Exception> streamReadPortSet = ApmToCcrAdapters.Read(fs, data, 0, data.Length); Arbiter.Activate(dq, Arbiter.Choice(streamReadPortSet, delegate(Int32 bytesRead) { Array.Resize(ref data, bytesRead); Msg("Read completed, bytes read={0}, data follows:{1}{2}", data.Length, Environment.NewLine, Encoding.ASCII.GetString(data)); }, delegate(Exception e) { Msg("Read failed, error={0}", e.Message); }));

In my AsyncStreamPortSetDemo method, I define a PortSet<Int32, Exception> variable (called streamReadPortSet) and in my ApmToCcrAdapters class I have an overload of the Read method that internally constructs a PortSet (and its Port object members) and returns a reference to this PortSet object.

The Arbiter class also has an overload of the Choice method that takes a PortSet<T0, T1> and allows you to pass two delegates to it. This overload of the Choice method internally calls the Choice method shown and discussed in the AsyncStreamDemo method section. The new code is simpler than the code shown in the AsyncStreamDemo method because you have one PortSet object instead of two Port objects and because Choice takes two delegates instead of having to call Arbiter's Receive method twice passing in a bunch of arguments.

I will use the PortSet in many of the following examples. I should also point out that the CCR defines several PortSet types with variations going all the way to PortSet<T0, T1, ..., T19>. However, PortSet<T0, T1> is by far the most commonly used.

SerialAsyncDemo The SerialAsyncDemo method demonstrates how to use a C# iterator to easily write code that performs several asynchronous operations sequentially (see Figure 6). While these operations are performed sequentially, no threads are blocked. This gives us the syntactical simplicity for sequential programming while allowing the application to scale to support hundreds of thousands of pending operations.

 Figure 6 SerialAsyncDemo

 

private static void SerialAsyncDemo(DispatcherQueue dq) { Arbiter.Activate(dq, Arbiter.FromIteratorHandler(SaveWebSiteToFile)); HitEnter(); } private static IEnumerator<ITask> SaveWebSiteToFile() { WebResponse webResponse = null; yield return Arbiter.Choice(ApmToCcrAdapters.GetResponse( WebRequest.Create("http://Wintellect.com")), delegate(WebResponse wr) { Msg("Got web data"); webResponse = wr; }, delegate(Exception e) { Msg("Failed to get web data"); }); if (webResponse == null) yield break; FileStream fs = new FileStream(@"WebData.html", FileMode.Create, FileAccess.Write, FileShare.Write, 8 * 1024, FileOptions.Asynchronous); using (fs) { Byte[] webData = new Byte[10000]; Int32 numbytes = webResponse.GetResponseStream().Read( webData, 0, webData.Length); Array.Resize(ref webData, numbytes); yield return Arbiter.Choice(ApmToCcrAdapters.Write(fs, webData, 0, webData.Length), delegate(EmptyValue ev) { Msg("Wrote web data to file"); }, delegate(Exception e) { Msg("Failed to write web data to file"); }); } }

The SaveWebSiteToFile method is a C# iterator method that first requests HTML data from a Web server and then writes this data to a file. To start the operation, I call Arbiter's static FromIteratorHandler method passing it the name of the C# iterator method.

Internally, the CCR will call into the iterator. Inside the iterator, I call one of my adapters to make an asynchronous Web request using a WebRequest object. My adapter returns a PortSet<WebResponse, Exception> indicating the two possible results. Then Choice is used to tell the CCR how to deal with each of the results. The Choice method returns an ITask object that is yield-returned back to the CCR. The CCR then activates this on the Port objects and the CCR's thread returns to the pool waiting for the I/O operation to complete.

If the Web request fails, a message is displayed and yield break executes telling the CCR that there are no more operations to be done. If the Web request completes successfully, a message is displayed indicating that the application got the Web data and then the iterator starts an asynchronous Write operation against a FileStream. Again, the result of the Choice method is yield-returned from the iterator back to the CCR, which then activates it.

If the write request completes, you'll either see a success or failure message and then the FileStream object will be closed. When the iterator exits, the CCR knows not to activate any more arbiters and the sequence of operations is complete.

Coordinating Several I/O Operations

The AsyncIOCoordination1 method demonstrates how to coordinate the results of several asynchronous operations. In this method, I have a loop that issues several asynchronous operations to request data from a Web server. As each request is made, a new PortSet<WebResponse, Exception> object is created and returned. I then activate a Choice arbiter on each PortSet object. Because of the way this code is written, as Web requests complete, the proper delegate will execute indicating which operations succeeded and which operations failed;

 

for (int n = 0; n < c_ImageUrls.Length; n++) { WebRequest webReq = WebRequest.Create(c_ImageUrls[n]); Int32 requestNum = n; // Capture n for the anonymous method Arbiter.Activate(dq, Arbiter.Choice(ApmToCcrAdapters.GetResponse(webReq), delegate(WebResponse webResp) { Byte[] data = new Byte[webResp.ContentLength]; webResp.GetResponseStream().Read(data, 0, data.Length); Msg("Request={0}, ResponseUri={1}", requestNum, webResp.ResponseUri); }, delegate(Exception e) { Msg("Request={0}, GetResponse failed", requestNum); })); }

The AsyncIOCoordination2 method demonstrates another way to coordinate the results of several asynchronous operations, as shown in Figure 7. In this method, I have a loop that issues several asynchronous operations to request data from a Web server. In this example, I am having all success results post to a single Port<WebResponse> object and all failure results get posted to a single Port<Exception> object. Then I activate an Interleave arbiter. An Interleave arbiter is similar to a reader/writer thread synchronization lock in that it can allow multiple threads access to a resource and it can also ensure mutual-exclusive access to a resource. When you call Arbiter's Interleave method, you pass it three arguments: a TeardownReceiverGroup object, an ExclusiveReceiverGroup object, and a ConcurrentReceiverGroup object. Each of these types has a constructor that takes an array of ReceiverTask objects.

 Figure 7 Coordinating Asychnronous Operations

 

Port<WebResponse> responsePort = null; Port<Exception> failurePort = null; for (Int32 n = 0; n < c_ImageUrls.Length; n++) { WebRequest webReq = WebRequest.Create(c_ImageUrls[n]); ApmToCcrAdapters.GetResponse(webReq, ref responsePort, ref failurePort); } Arbiter.Activate(dq, Arbiter.Interleave( new TeardownReceiverGroup( Arbiter.Receive(false, failurePort, delegate(Exception e) { Msg("At least 1 GetResponse failed"); })), new ExclusiveReceiverGroup(), new ConcurrentReceiverGroup( Arbiter.Receive(true, responsePort, delegate(WebResponse response) { Byte[] data = new Byte[response.ContentLength]; response.GetResponseStream().Read(data, 0, data.Length); Msg("RequestUri={0}", response.ResponseUri); }))));

In my AsyncIOCoordination2, Interleave is passed a TeardownReceiverGroup object that has just one ReceiverTask object. This tells the CCR to unregister the entire Interleave from all Port objects as soon as a failure is detected on the failure Port object. In other words, if any Web request operation fails, the CCR shouldn't process any more successfully completed operations. I pass no ReceiverTask object to the ExclusiveReceiverGroup constructor because my demo has no code that must execute exclusively. Finally, I pass to Interleave a ConcurrentReceiverGroup that has one ReceiverTask object. This object tells the CCR how to process each Web request operation that completes successfully. Because this ReceiverTask object is in the concurrent group, multiple Dispatcher threads can execute this code simultaneously should multiple Web requests complete at (or near) the same time.

By the way, if the code that executed in response to a completed Web request were to touch shared data, you could use a thread synchronization lock (like a Monitor or Mutex) around the code block; however, I would discourage this. A better way to ensure that only one thread at a time will access the shared data would be to move the ReceiverTask object from the ConcurrentReceiverGroup constructor into the ExclusiveReceiverGroup constructor. This way, the CCR will ensure for you that only one thread at a time can execute the code to process a successful Web request. As you can see, the coordination aspects of the CCR are quite powerful and easy to use once you get familiar with them. You will have to experiment with them a bit before you really see the flexibility and control available to you.

The AsyncIOCoordination3 method demonstrates yet another way to coordinate the results of several asynchronous operations. This method also demonstrates how to introduce a timer into the coordination. In this method, I create three ports: responsePort, a Port<WebResponse> used for success results; failurePort, a Port<Exception> used for failure results; and timeoutPort, a Port<DateTime> used for timeout.

Like the previous two examples, I have a loop that issues several asynchronous operations to request data from a Web server (see Figure 8). In this example, I am having all success results post to responsePort and all failure results post to failurePort. After initiating all of the asynchronous I/O requests, I then call DispatcherQueue's EnqueueTimer method. This call to EnqueueTimer tells the DispatcherQueue to wait 2,000 milliseconds and then to post the current date and time into the timeoutPort.

 Figure 8 Coordinating Operations with a Timer

 

Port<WebResponse> responsePort = null; Port<Exception> failurePort = null; Port<DateTime> timeoutPort = new Port<DateTime>(); for (Int32 n = 0; n < c_ImageUrls.Length; n++) { WebRequest webReq = WebRequest.Create(c_ImageUrls[n]); ApmToCcrAdapters.GetResponse(webReq, ref responsePort, ref failurePort); } dq.EnqueueTimer(TimeSpan.FromMilliseconds(2000), timeoutPort); Arbiter.Activate(dq, Arbiter.Choice( Arbiter.Receive(false, failurePort, delegate(Exception e) { Msg("At least 1 GetResponse failed"); }), Arbiter.Receive(false, timeoutPort, delegate(DateTime dt) { Msg( "Some requests did not completed within 2 seconds."); }), Arbiter.MultipleItemReceive(false, responsePort, c_ImageUrls.Length, delegate(WebResponse[] responses) { foreach (WebResponse response in responses) { Byte[] data = new Byte[response.ContentLength]; response.GetResponseStream().Read( data, 0, data.Length); Msg("ResponseUrl={0}", response.ResponseUri); } })));

I next activate a Choice arbiter which ensures that one and only one of the three arbiters passed to it will execute. The first Receive arbiter indicates what code should execute should any Web request fail. The second Receive arbiter indicates what code should execute should the 2,000 milliseconds expire while waiting for all the Web requests. The third arbiter, a MultipleItemReceive, indicates what method to execute after all of the Web requests have completed. If all the Web requests complete at the same time that the 2,000 milliseconds expires, Choice ensures that only one of the methods will execute; you do not have to handle potential race conditions in your code.

Conclusion

The CCR is a CLR library that provides a consistent and scalable way to program asynchronous operations and coordinate among multiple responses. Framework Class Library (FCL) classes that already support the CLR's asynchronous programming model (such as Stream's BeginRead and BeginWrite) methods can easily be wrapped, allowing existing types to integrate with the CCR so that complex failure handling and coordination patterns can be coded in a robust, reliable, and concise way. The use of C# iterators for scheduling operations allows sequential programming without blocking OS threads, thus enabling scaling without sacrificing the simplicity of sequential code.

Quietly tucked away inside the Microsoft Robotics Studio, the CCR is a potent technology that could be applied for many other applications. For more information about availability of the CCR and other interesting technologies included in the Microsoft Robotics Studio, see msdn.microsoft.com/robotics and the Channel 9 video on CCR development at channel9.msdn.com/Showpost.aspx?postid=219308.

Send your questions and comments for Jeffrey to  mmsync@microsoft.com.

Jeffrey Richter is a cofounder of Wintellect, a training and consulting firm. He is the author of several books, including CLR via C# (Microsoft Press, 2006). Jeffrey is also a contributing editor to MSDN Magazine and has been consulting with Microsoft since 1990