[转]Marshaling a SAFEARRAY of Managed Structures by P/Invoke Part 1.

1. Introduction.

1.1 I have previously written about exchanging
SAFEARRAYs of managed structures with unmanaged code via COM
interop.

1.2 In this new series of articles, I shall expound on
the exchange of such SAFEARRAYs via P/Invoke.

1.3 I have arranged this series of articles
into multiple parts. In the first 3 parts, I shall show simple marshaling
of an array of C# structure to and from unmanaged code via APIs exported from a
DLL. The marshaling of the C# array is done with the use of
SAFEARRAYs.

1.4 In later parts, I will demonstrate how
to marshal a structure that contains an array of structures.

1.5 Before the end of this series, I will up the
ante even further by demonstrating how to marshal a structure that
contains an array of structures that each contains an array of structures.
Sounds confusing I know but I have seen questions on MSDN forums that
are based on this type of convoluted scenario. It is also something which I have
long wanted to expound on as it demonstrates the power and consistency
of SAFEARRAYs.

1.6 What I intend to do is to explain the principles of
the marshaling process as well as the rudiments of using SAFEARRAYs to contain
structures. Even structures that contain more SAFEARRAYs. Important information
on the avoidance of memory crash and leakage will also be presented.

1.7 I shall be using C# to develop the managed code and
the umanaged code shall be written in C++.

1.8 Note that throughout this series of artciles, we
shall be working only with single-dimensional managed arrays and
SAFEARRAYs.

2. Why Use SAFEARRAYs ?

2.1 In various articles that I have previously
written in my blog, I have emphasized the advantage of using SAFEARRAYs to
represent managed arrays, these are summarized below :

• SAFEARRAYs, just like their managed array counterparts,
  intrinsically holds information on the data type of the elements that it
  contains.
• SAFEARRAYs also intrinsically holds information on the
  number of dimensions and the number of elements per dimension of its
  array.
• SAFEARRAYs are created and destroyed using standard
  Windows APIs like SafeArrayCreate() and SafeArrayDestroy(). Management APIs like
  SafeArrayGetLBound() and SafeArrayGetUBound() are also available. This makes it
  suitable for use in all unmanaged language.
• The last point also makes SAFEARRAYs accessible and
  manageable by the CLR. This is why the use of SAFEARRAYs is the preferred
  way to exchange arrays between managed and unmanaged code. Especially so in
  cases where the size of the array may change during a call to unmanaged
  functions.

2.2 The SAFEARRAY is thus the best unmanaged type that
matches a managed array.

3. The Test Structures.

3.1 For the purpose of this article, I have defined
the following basic C# structure that will be used throughout the series of
blogs :

public struct TestStructure
{
    public Int32 m_integer;
    public double m_double;
    public string m_string;
}

3.2 As mentioned in a previous article I wrote (see
points 2 and 3 of Marshaling
a SAFEARRAY of Managed Structures by COM Interop Part 1), a structure that
is to be contained inside a SAFEARRAY must be a COM User-Defined Type
(UDT).

3.3 To enable a managed structure to be transformed into
a COM UDT, we use various .NET attributes to describe elements of the managed
structure :

[ComVisible(true)]
[StructLayout(LayoutKind.Sequential, Pack = 1)]
[Guid("1979BCD7-1062-44d8-B3FC-A2686C61E715")]
public struct TestStructure
{
    [MarshalAs(UnmanagedType.I4)]
    public Int32 m_integer;
    [MarshalAs(UnmanagedType.R8)]
    public double m_double;
    [MarshalAs(UnmanagedType.BStr)]
    public string m_string;
}

Here, I used the ComVisibleAttribute,
StructLayoutAttribute, GuidAttribute and various MarshalAsAttributes to describe
TestStructure so that when we later use TLBEXP.EXE to
generate a type library for the assembly that contains the C# structure, a
COM UDT will be properly defined inside the type library.

3.4 For more information, please refer to points 2 and 3
of Marshaling
a SAFEARRAY of Managed Structures by COM Interop Part 1.

3.5 In order to expose TestStructure as a COM UDT,
we must first define it inside a C# assembly (DLL or EXE) and then create a
type library from the C# assembly using the Type Library Exporter (TLBEXP.EXE).

3.6 For the purpose of this article, I contained the
definition of TestStructure inside a C# project (named CSConsoleApp) that
produces an EXE assembly. For full source codes, please refer to point
6.8.

4. Type Library Creation.

4.1 As mentioned previously, to expose
TestStructure to the unmanaged world by way of a COM type library, we use
the Type Library Exporter (TLBEXP.EXE).

4.2 The following is a typical command line showing how
we can use TLBEXP.EXE to
produce a type library from the COM-visible contents of a managed assembly
:

tlbexp CSConsoleApp.exe /out:CSConsoleApp.tlb

4.3 When we examine CSConsoleApp.tlb via OLEVIEW.EXE,
we can see the following definition of TestStructure :

// Generated .IDL file (by the OLE/COM Object Viewer)
// 
 
[
  uuid(4E765D3B-C1F5-4CDE-8095-1E9614E0AE3F),
  version(1.0)
]
library CSConsoleApp
{
    // TLib :     // Forward declare all types defined in this typelib
    typedef
    [
	uuid(1979BCD7-1062-44D8-B3FC-A2686C61E715), version(1.0),
    	custom({0F21F359-AB84-41E8-9A78-36D110E6D2F9}, "CSConsoleApp.TestStructure")
    ]
    struct tagTestStructure
    {
	long m_integer;
	double m_double;
	BSTR m_string;
    } TestStructure;
};

Note that the GUID associated with TestStructure is
exactly that which we used in point 2.3. The data type for each of the member
fields also correspond to how we defined them using
MarshalAsAttributes.

4.4 Now that we have produced CSConsoleApp.tlb,
we can use it to create an unmanaged DLL that exposes APIs that exchanges arrays
of managed structures via SAFEARRAYs.

4.5 The type library is used by unmanaged code as
that references to COM entities (e.g. the TestStructure struct) can be resolved.
In the C++ DLL project which will be described in the next section, we will be
#importing this type library.

4.6 Note that in further installments of this series of
articles, I shall be augmenting the source codes of CSConsoleApp.exe with
more structure definitions and will be re-creating CSConsoleApp.tlb.

5. Unmanaged API that takes a SAFEARRAY of
TestStructure.

5.1 In this section, I shall write a C++ exported
function that can be imported into a C# application. For the purpose of this
article, I have written this exported function as part of a DLL
project the output of which is UnmanagedDll.dll.
In order to adequately reference the TestStructure struct in the exported
function, the CSConsoleApp.tlb type
library must be #imported into the source codes, e.g. :

#import "CSConsoleApp.tlb" raw_interfaces_only no_implementation
using namespace CSConsoleApp;

5.2 The exported function that we will expose to
C# takes as input a pointer to a SAFEARRAY of TestStructure structures,
performs some basic tests to ascertain its characteristics (e.g. that it
contains VT_RECORD types and that the GUID of the structures contained in the
SAFEARRAY is as expected) and then displays the values of members of each
structure.

5.3 The following is a full code listing of this
function :

extern "C" __declspec(dllexport) void __stdcall SetArrayOfTestStructure
(
	/*[in]*/ SAFEARRAY* pSafeArrayOfTestStructure
)
{
	// Call the SafeArrayGetVartype() to determine
	// the VARTYPE of the contained elements.
	VARTYPE vt;	
 
	// The value is VT_RECORD (i.e. 36).
	SafeArrayGetVartype(pSafeArrayOfTestStructure, &vt);
 
	_ASSERT(vt == VT_RECORD);
 
	// Get the IRecordInfo object associated with the structure.
	IRecordInfoPtr spIRecordInfo = NULL;
	SafeArrayGetRecordInfo(pSafeArrayOfTestStructure, &spIRecordInfo);
 
	// Get the GUID associated with the IRecordInfo object.
	// This is the same GUID associated with the structure,
	// i.e. 1979bcd7-1062-44d8-b3fc-a2686c61e715.
	GUID guid;
	spIRecordInfo -> GetGuid(&guid);
 
	_ASSERT(guid == __uuidof(TestStructure));
 
	// Obtain bounds information of the SAFEARRAY.
	long lLbound = 0;
	long lUbound = 0;
 
	SafeArrayGetLBound(pSafeArrayOfTestStructure, 1, &lLbound);
	SafeArrayGetUBound(pSafeArrayOfTestStructure, 1, &lUbound);
	long lDimSize = lUbound - lLbound + 1;		
 
	printf ("Size of SAFEARRAY : [%d].\r\n", lDimSize);
 
	// Obtain a copy of each structure contained
	// in the SAFEARRAY.
	for (int i = 0; i < lDimSize; i++)
    	{
		long rgIndices[1];
		TestStructure value;
		// Note that we must set all fields
		// inside "value" to zero.
		// This is important because
		// SafeArrayGetElement() will first
		// clear all members of value.
		// For the "m_string" BSTR member,
		// it will call the ::SysFreeString()
		// API. Now if "m_string" contains
		// random data, it will result in
		// a crash. If it is NULL, it is OK.
		memset(&value, 0, sizeof(TestStructure));
		// SafeArrayGetElement() will return
		// a copy of the relevant element in
		// the structure. Hence it is important
		// that when "value" is no longer required,
		// its members be cleared. This is especially
		// so for members which point to allocated
		// memory (e.g. BSTRs).
		rgIndices[0] = i;
		SafeArrayGetElement
		(
			pSafeArrayOfTestStructure,
			rgIndices,
			(void FAR*)&value
		);
 
		printf ("TestStructure[%d].m_integer : [%d]\r\n", i, value.m_integer);
		printf ("TestStructure[%d].m_double  : [%f]\r\n", i, value.m_double);
		printf ("TestStructure[%d].m_string  : [%S]\r\n", i, value.m_string);
		// We clear the members of "value"
		// which includes the "m_string" BSTR member.
		spIRecordInfo -> RecordClear((PVOID)&value);
	}
 
	// Note that the input pointer to the SAFEARRAY,
	// i.e. "pSafeArrayOfTestStructure", must not
	// be destroyed. It is to be treated as READ-ONLY.
	// By the way, the owner of this SAFEARRAY is
	// the Interop Marshaler and it will destroy it
	// when this API returns to managed code.
	//::SafeArrayDestroy(pSafeArrayOfTestStructure);
	//pSafeArrayOfTestStructure = NULL;
}

The following is a general synopsis of this
function :

• It uses the SafeArrayGetVartype() Windows API to
  determine the Variant Type of the elements stored inside the
  SAFEARRAY.
• This Variant Type must turn out to be
  VT_RECORD.
• It then uses the SafeArrayGetRecordInfo() API to obtain
  a pointer to the IRecordInfo interface associated with the UDT contained inside
  the SAFEARRAY.
• The IRecordInfo interface is then used to obtain the
  GUID of the UDT.
• This GUID is then checked to ensure that it matches that
  for TestStructure.
• The SafeArrayGetLBound() and SafeArrayGetUBound() APIs
  are then used to determine the number of elements contained inside the
  array.
• The function then loops through the elements of the
  array and displays the field values of each TestStructure.
• There are two very important memory-related
  activities performed as part of this loop : the clearing of a TestStructure
  struct before it is used to obtain a copy of an element of the SAFEARRAY
  and the memory clearance of the same structure when it is no longer
  needed.
• These will be explained in their own sections below
  later on (see sections 7 and 8).
• Then finally, before the function finishes, make sure
  that the input SAFEARRAY is not destroyed
  with a call to SafeArrayDestroy(). I shall also explain this in greater detail
  in a separate section (section 9).

6. Example C# Call to
SetArrayOfTestStructure().

6.1  The following shows how the
SetArrayOfTestStructure() API should be declared in a C# program :

[DllImport("UnmanagedDll.dll", CallingConvention = CallingConvention.StdCall)]
private static extern void SetArrayOfTestStructure
(
  [In] [MarshalAs(UnmanagedType.SafeArray, SafeArraySubType = VarEnum.VT_RECORD)]
  TestStructure[] SafeArrayOfTestStructure
);

The parameter to this function is a managed array of
TestStructure structs. We are now working in C# code and so, at this level, we
must work with managed arrays.

Now note the use of the various attributes :

• The InAttribute is used to indicate to the interop
  marshaler that the “SafeArrayOfTestStructure” parameter is to be marshaled
  single-directionally “into” the function.
• This also indicates to the interop marshaler that
  whatever form the “SafeArrayOfTestStructure” parameter takes when it enters the
  unmanaged function, it will be treated as read-only by the
  function.
• The MarshalAsAttribute is used to indicate to the
  interop marshaler that the “SafeArrayOfTestStructure” parameter is to
  be marshaled as a SAFEARRAY.
• Furthermore, the “SafeArraySubType” field for the
  MarshalAsAttribute, being equal to “VarEnum.VT_RECORD”, indicates to the interop
  marshaler that the SAFEARRAY must contain UDTs.

6.2 The following is a sample C# function that makes a
call to SetArrayOfTestStructure() :

static void DoTest_SetArrayOfTestStructure()
{
    TestStructure[] SafeArrayOfTestStructure = new TestStructure[10];
 
    for (int i = 0; i < SafeArrayOfTestStructure.Length; i++)
    {
        SafeArrayOfTestStructure[i].m_integer = i;
        SafeArrayOfTestStructure[i].m_double = (double)i;
        SafeArrayOfTestStructure[i].m_string = string.Format("Hello World [{0}]", i);
    }
 
    SetArrayOfTestStructure(SafeArrayOfTestStructure);
}

The following is a synopsis :

• It allocates a managed array of 10 TestStructure
  structures.
• It loops through the elements of the array and assigns
  simple field values to each.
• It then calls the SetArrayOfTestStructure() API,
  passing the array as parameter.

6.3 What happens under the covers when the managed
array “SafeArrayOfTestStructure” gets passed to the SetArrayOfTestStructure()
API is that the interop marshaler will internally create a SAFEARRAY and fill it
with UDTs each of which matches a corresponding managed TestStructure
structure inside “SafeArrayOfTestStructure”.

6.4 The interop marshaler is also able to dynamically
generate a COM IRecordInfo object that will be associated with the
UDT-equivalent of TestStructure. A pointer to the IRecordInfo interface of this
object will be referenced inside the SAFEARRAY.

6.5 A pointer to the SAFEARRAY is then passed to
the SetArrayOfTestStructure() API as an “in” (read-only) parameter.

6.6 More about this SAFEARRAY will be expounded in the
next few sections which serve as important advisories.

6.7 At runtime, the C# function
DoTest_SetArrayOfTestStructure(), in conjunction with the call to
SetArrayOfTestStructure(), will produce the following expected output
:

TestStructure[1].m_double  : [1.000000]
TestStructure[1].m_string  : [Hello World [1]]
TestStructure[2].m_integer : [2]
TestStructure[2].m_double  : [2.000000]
TestStructure[2].m_string  : [Hello World [2]]
TestStructure[3].m_integer : [3]
TestStructure[3].m_double  : [3.000000]
TestStructure[3].m_string  : [Hello World [3]]
TestStructure[4].m_integer : [4]
TestStructure[4].m_double  : [4.000000]
TestStructure[4].m_string  : [Hello World [4]]
TestStructure[5].m_integer : [5]
TestStructure[5].m_double  : [5.000000]
TestStructure[5].m_string  : [Hello World [5]]
TestStructure[6].m_integer : [6]
TestStructure[6].m_double  : [6.000000]
TestStructure[6].m_string  : [Hello World [6]]
TestStructure[7].m_integer : [7]
TestStructure[7].m_double  : [7.000000]
TestStructure[7].m_string  : [Hello World [7]]
TestStructure[8].m_integer : [8]
TestStructure[8].m_double  : [8.000000]
TestStructure[8].m_string  : [Hello World [8]]
TestStructure[9].m_integer : [9]
TestStructure[9].m_double  : [9.000000]
TestStructure[9].m_string  : [Hello World [9]]

6.8 For the purpose of this article, I have
include the source codes for DoTest_SetArrayOfTestStructure() into the
same C# console application CSConsoleApp which also contained the
definition of TestStructure. The the main source file of this
application (Program.cs)
is listed below :

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Runtime.InteropServices;
 
namespace CSConsoleApp
{
    [ComVisible(true)]
    [StructLayout(LayoutKind.Sequential, Pack = 1)]
    [Guid("1979BCD7-1062-44d8-B3FC-A2686C61E715")]
    public struct TestStructure
    {
        [MarshalAs(UnmanagedType.I4)]
        public Int32 m_integer;
        [MarshalAs(UnmanagedType.R8)]
        public double m_double;
        [MarshalAs(UnmanagedType.BStr)]
        public string m_string;
    }
 
    class Program
    {
        [DllImport("UnmanagedDll.dll", CallingConvention = CallingConvention.StdCall)]
        private static extern void SetArrayOfTestStructure
	(
		[In] [MarshalAs(UnmanagedType.SafeArray, SafeArraySubType = VarEnum.VT_RECORD)]
		TestStructure[] SafeArrayOfTestStructure
	);
 
        static void DoTest_SetArrayOfTestStructure()
        {
            TestStructure[] SafeArrayOfTestStructure = new TestStructure[10];
 
            for (int i = 0; i < SafeArrayOfTestStructure.Length; i++)
            {
                SafeArrayOfTestStructure[i].m_integer = i;
                SafeArrayOfTestStructure[i].m_double = (double)i;
                SafeArrayOfTestStructure[i].m_string = string.Format("Hello World [{0}]", i);
            }
 
            SetArrayOfTestStructure(SafeArrayOfTestStructure);
        }
 
        static void Main(string[] args)
        {
            DoTest_SetArrayOfTestStructure();
        }
    }
}

7. Always Clear the Contents of a Receiving UDT Before A
Call To SafeArrayGetElement().

7.1 Before a call to SafeArrayGetElement(), always
ensure that the contents of the structure (that is used to receive a copy of the
UDT element inside the SAFEARRAY) is cleared.

7.2 This can be seen in the loop contained inside the
SetArrayOfTestStructure() function as shown in point 5.3 :

		long rgIndices[1];
		TestStructure value;
		// Note that we must set all fields
		// inside "value" to zero.
		// This is important because
		// SafeArrayGetElement() will first
		// clear all members of value.
		// For the "m_string" BSTR member,
		// it will call the ::SysFreeString()
		// API. Now if "m_string" contains
		// random data, it will result in
		// a crash. If it is NULL, it is OK.
		memset(&value, 0, sizeof(TestStructure));
		// SafeArrayGetElement() will return
		// a copy of the relevant element in
		// the structure. Hence it is important
		// that when "value" is no longer required,
		// its members be cleared. This is especially
		// so for members which point to allocated
		// memory (e.g. BSTRs).
		rgIndices[0] = i;
		SafeArrayGetElement
		(
			pSafeArrayOfTestStructure,
			rgIndices,
			(void FAR*)&value
		);

Here the structure to receive a copy of a UDT is “value”
(of type TestStructure) and we first clear it by using memset().

7.3 As mentioned in the comments, the
SafeArrayGetElement() API will first clear all members of the receiving
structure before assign them with copies from the fields of the corresponding
structure inside the SAFEARRAY.

7.4 Hence any member which is a referenced type will
first be released. For example, SysFreeString() will be called on a BSTR. If the
BSTR member points to random data, the call to SysFreeString() will result in a
crash. But if the BSTR member is set to zero, the call to SysFreeString() will
go through fine.

8. Clear the UDT Copied From A SAFEARRAY
When It Is No Longer Required.

8.1 Note that after a UDT has been copied from a
SAFEARRAY via SafeArrayGetElement(), it needs to be cleared when no longer
required.

8.2 This is because the UDT is a copy of its
corresponding element from the SAFEARRAY. When the SAFEARRAY is destroyed, any
UDT acquired through SafeArrayGetElement() is not automatically destroyed. It
is, after all, a copy of one of the UDTs contained in the SAFEARRAY.

8.3 Hence always call IRecordInfo::RecordClear() on the
UDT copy when it is no longer required as shown in the code of point 5.3
:

		// We clear the members of "value"
		// which includes the "m_string" BSTR member.
		spIRecordInfo -> RecordClear((PVOID)&value);

8.4 Alternatively, you may also manually perform the
clearing by making explicit calls to clearance functions (e.g. SysFreeString()
or IUnknown::Release()) on specific members.

8.5 Failure to do so will result in memory
leakage.

9. Do Not Destroy An “In”
SAFEARRAY.

9.1 SAFEARRAYs which are input as “in” parameters to a
function must not be destroyed (e.g. via SafeArrayDestroy()).

9.2 In the case of the
SetArrayOfTestStructure() API, the “pSafeArrayOfTestStructure” parameter is an
“in” or read-only parameter. It remains owned by the caller which is the interop
marshaler.

9.3 As such, SetArrayOfTestStructure() must not
destroy it. It will be destroyed by the interop marshaler when the right time
comes (i.e. when control returns to managed code).

10. In Conclusion.

10.1 Here in part 1 I have demonstrated a basic transfer
ot a managed array of structures to unmanaged code.

10.2 The transfer is done through the use of a
SAFEARRAY.

10.3 Important steps to ensure the receipt of a correct
SAFEARRAY is shown through actual code (point 5.3).

10.4 Important advisories on memory safety are also
discussed (sections 7, 8 and 9).

10.5 In the next installment of this series of articles,
I shall demonstrate how to return a SAFEARRAY of UDTs from an unmanaged
function which will be transformed into a managed array.