MPI: Everyday Datatypes

Heterogeneous computing requires that the data constituting a message be typed or described somehow so that its machine representation can be converted between computer architectures. MPI can thoroughly describe message datatypes, from the simple primitive machine types to complex structures, arrays and indices.

MPI messaging functions accept a datatype parameter, whose C typedef is MPI_Datatype:

    MPI_Send(void* buf, int count, MPI_Datatype datatype,
	    int dest, int tag, MPI_Comm comm);

Basic Datatypes

    MPI_CHAR (char)
    MPI_INT (int)
    MPI_FLOAT (float)
    MPI_DOUBLE (double)

For messages consisting of a homogeneous, contiguous array of basic datatypes, this is the end of the datatype discussion. For messages that contain more than one datatype or whose elements are not stored contiguously in memory, something more is needed.

Strided Vector

An MPI derived datatype is a good solution for a non-contiguous data structure. A code fragment to derive an appropriate datatype matching this strided vector and then transmit the last column is listed below:

#include <mpi.h>

{
    float		mesh[10][20];
    int			dest, tag;
    MPI_Datatype	newtype;

/*
 * Do this once.
 */
    MPI_Type_vector(10,		/* # column elements */
	    1,			/* 1 column only */
	    20,			/* skip 20 elements */
	    MPI_FLOAT,		/* elements are float */
	    &newtype);		/* MPI derived datatype */

    MPI_Type_commit(&newtype);
/*
 * Do this for every new message.
 */
    MPI_Send(&mesh[0][19], 1, newtype,
	    dest, tag, MPI_COMM_WORLD);
}

MPI_Type_commit( ) separates the datatypes you really want to save and use from the intermediate ones that are scaffolded on the way to some very complex datatype.

A nice feature of MPI derived datatypes is that once created, they can be used repeatedly with no further set-up code. MPI has many other derived datatype constructors.

C Structure

    struct {
	int		lineno;
	char		pixels[1024];
    } scanline;

#include <mpi.h>

{
    unsigned int	membersize, maxsize;
    int			position;
    int			dest, tag;
    char		*buffer;
/*
 * Do this once.
 */
    MPI_Pack_size(1, 		/* one element */
	    MPI_INT,		/* datatype integer */
	    MPI_COMM_WORLD,	/* consistent comm. */
	    &membersize);	/* max packing space req'd */

    maxsize = membersize;
    MPI_Pack_size(1024, MPI_CHAR, MPI_COMM_WORLD, &membersize);
    maxsize += membersize;
    buffer = malloc(maxsize);
/*
 * Do this for every new message.
 */
    position = 0;

    MPI_Pack(&scanline.lineno,	/* pack this element */
	    1,			/* one element */
	    MPI_INT,		/* datatype int */
	    buffer,		/* packing buffer */
	    maxsize,		/* buffer size */
	    &position,		/* next free byte offset */
	    MPI_COMM_WORLD);	/* consistent comm. */

    MPI_Pack(scanline.pixels, 1024, MPI_CHAR,
	    buffer, maxsize, &position, MPI_COMM_WORLD);

    MPI_Send(buffer, position, MPI_PACKED,
	    dest, tag, MPI_COMM_WORLD);
}

A buffer is allocated once to contain the size of the packed structure. The size must be computed because of implementation dependent overhead in the message. Variable sized messages can be handled by allocating a buffer large enough for the largest possible message. The position parameter to MPI_Pack( ) always returns the current size of the packed buffer.

A code fragment to unpack the message, assuming a receive buffer has been allocated, is listed below:

{
    int             src;
    int             msgsize;
    MPI_Status      status;

    MPI_Recv(buffer, maxsize, MPI_PACKED,
	    src, tag, MPI_COMM_WORLD, &status);

    position = 0;
    MPI_Get_count(&status, MPI_PACKED, &msgsize);

    MPI_Unpack(buffer,		/* packing buffer */
	    msgsize,		/* buffer size */
	    &position,		/* next element byte offset */
	    &scanline.lineno,	/* unpack this element */
	    1,			/* one element */
	    MPI_INT,		/* datatype int */
	    MPI_COMM_WORLD);	/* consistent comm. */

    MPI_Unpack(buffer, msgsize, &position,
	    scanline.pixels, 1024, MPI_CHAR, MPI_COMM_WORLD);
}