A while ago I posted about having a difficult time with complex marshaling. What I didn’t post, however, was any details about the solution–and now that the topic of custom marshaling has become the bane of a few co-worker’s existences, I figured I’d post some details.

Here’s the problem. You have a struct (for example, the WAVEFORMATEX structure) that you need to marshal to a Win32 API call (for example, acmStreamOpen). Most of the time, when marshaling, you can get away with at best directly passing types (such as int, or a reference to a struct (e.g. ref ACMSTREAMHEADER) and at less-than-best using System.Runtime.InteropServices.Marshal helper methods to convert between managed and unmanaged types. Sometimes, though, you have to use the worst-case scenario: custom marshaling. The WAVEFORMATEX struct looks like this:

typedef struct {
     WORD  wFormatTag;
     WORD  nChannels;
     DWORD nSamplesPerSec;
     DWORD nAvgBytesPerSec;
     WORD  nBlockAlign;
     WORD  wBitsPerSample;
     WORD  cbSize;
} WAVEFORMATEX;

where cbSize defines the size of additional data describing the waveform (e.g. coefficients, etc). If uncompressed PCM waveforms were all we had to deal with, then life would be good. But if you want to handle compressed formats (such as ADPCM or µ-law), the data structure looks different, like this:

typedef struct adpcmwaveformat_tag {
     WAVEFORMATEX wfx;
     WORD wSamplesPerBlock;
     WORD wNumCoef;
      ADPCMCOEFSET aCoef[];
} ADPCMWAVEFORMAT;

typedef struct adpcmcoef_tag {
     short iCoef1;
     short iCoef2;
} ADPCMCOEFSET;

Now we’ve got an extension of the WAVEFORMATEX struct that includes a dynamic array of ADPCMCOEFSET structs, with the length of the array specified as a field in the ADPCMWAVEFORMAT struct. Cool? Not cool if you want to marshal this beast and keep from using unsafe code. Unsafe code isn’t necessarily bad, but without a custom security policy in place it typically requires Full Trust to run, and if poorly understood it can introduce a myriad of problems.

In my next post I’ll talk about how to marshal this structure into the P/Invoke declaration for acmStreamOpen. To whet your appetite, here is the DllImport specification (and a hint as to how we’re going to accomplish the marshaling).

[DllImport("Msacm32.dll")]
public static extern uint acmStreamOpen(ref IntPtr phas,
          int had,
          [MarshalAs(UnmanagedType.CustomMarshaler, MarshalTypeRef=typeof(WAVEFORMATEXMarshaler))] WAVEFORMATEX pwfxSrc,
          [MarshalAs(UnmanagedType.CustomMarshaler, MarshalTypeRef=typeof(WAVEFORMATEXMarshaler))] WAVEFORMATEX pwfxDst,
           IntPtr pwfltr,
           IntPtr dwCallback,
           IntPtr dwInstance,
           uint fdwOpen);