Warning

This is the documentation for the development version of DTK. There may be significant differences from the latest stable release. Please follow this link if you are looking for DTK 3.0

DataTransferKit API

C++ API

User Function Registry

template <class Scalar>
class DataTransferKit::UserFunctionRegistry

Registry for user functions.

The registry is the mechanism by which user functions and the data to be called with those user functions are registered with DTK.

Template Parameters
  • Scalar: Scalar type of the fields this user function implementation will provide.

Set Geometry

void setNodeListSizeFunction(NodeListSizeFunction &&func, std::shared_ptr<void> user_data = nullptr)

Node list size function.

void setNodeListDataFunction(NodeListDataFunction &&func, std::shared_ptr<void> user_data = nullptr)

Node list data function.

void setBoundingVolumeListSizeFunction(BoundingVolumeListSizeFunction &&func, std::shared_ptr<void> user_data = nullptr)

Bounding volume list size function.

void setBoundingVolumeListDataFunction(BoundingVolumeListDataFunction &&func, std::shared_ptr<void> user_data = nullptr)

Bounding volume list data function.

void setPolyhedronListSizeFunction(PolyhedronListSizeFunction &&func, std::shared_ptr<void> user_data = nullptr)

Polyhedron list size function.

void setPolyhedronListDataFunction(PolyhedronListDataFunction &&func, std::shared_ptr<void> user_data = nullptr)

Polyhedron list data function.

void setCellListSizeFunction(CellListSizeFunction &&func, std::shared_ptr<void> user_data = nullptr)

Cell list size function.

void setCellListDataFunction(CellListDataFunction &&func, std::shared_ptr<void> user_data = nullptr)

Cell list data function.

void setBoundarySizeFunction(BoundarySizeFunction &&func, std::shared_ptr<void> user_data = nullptr)

Boundary data function.

void setBoundaryDataFunction(BoundaryDataFunction &&func, std::shared_ptr<void> user_data = nullptr)

Boundary data function.

void setAdjacencyListSizeFunction(AdjacencyListSizeFunction &&func, std::shared_ptr<void> user_data = nullptr)

Adjacency list size function.

void setAdjacencyListDataFunction(AdjacencyListDataFunction &&func, std::shared_ptr<void> user_data = nullptr)

Adjacency list data function.

Set Degree-of-freedom Maps

void setDOFMapSizeFunction(DOFMapSizeFunction &&func, std::shared_ptr<void> user_data = nullptr)

Single dofs per object dof map size.

void setDOFMapDataFunction(DOFMapDataFunction &&func, std::shared_ptr<void> user_data = nullptr)

Single dofs per object dof map data.

void setMixedTopologyDOFMapSizeFunction(MixedTopologyDOFMapSizeFunction &&func, std::shared_ptr<void> user_data = nullptr)

Multiple dofs per object dof map size.

void setMixedTopologyDOFMapDataFunction(MixedTopologyDOFMapDataFunction &&func, std::shared_ptr<void> user_data = nullptr)

Multiple dofs per object dof map data.

Set Fields

void setFieldSizeFunction(FieldSizeFunction<Scalar> &&func, std::shared_ptr<void> user_data = nullptr)

Field size.

void setPullFieldDataFunction(PullFieldDataFunction<Scalar> &&func, std::shared_ptr<void> user_data = nullptr)

Pull field.

void setPushFieldDataFunction(PushFieldDataFunction<Scalar> &&func, std::shared_ptr<void> user_data = nullptr)

Push field.

void setEvaluateFieldFunction(EvaluateFieldFunction<Scalar> &&func, std::shared_ptr<void> user_data = nullptr)

Evaluate field.

Public Types

template<>
using UserImpl = std::pair<CallableObject, std::shared_ptr<void>>

User implementation.

User Application

template <class Scalar, class ParallelModel>
class DataTransferKit::UserApplication

High-level interface to user applications.

The user application provides a high-level interface to compose DTK input data structures and push and pull field data to and from the application through sequences of user function calls.

Template Parameters
  • Scalar: Scalar type of the fields this user application will provide.
  • ParallelModel: The DTK parallel model indicating the on-node parallelism of the user application. Indicates where data will be allocated.

Type Aliases

template<>
using MemorySpace = typename ParallelModel::memory_space

Public Functions

UserApplication(const std::shared_ptr<UserFunctionRegistry<Scalar>> &user_functions)

Constructor.

auto getNodeList()

Get a node list from the application.

auto getBoundingVolumeList()

Get a bounding volume list from the application.

auto getPolyhedronList()

Get a polyhedron list from the application.

auto getCellList()

Get a cell list from the application.

template <class ListType>
void getBoundary(ListType &list)

Get a boundary from the application and put it in the given list.

template <class ListType>
void getAdjacencyList(ListType &list)

Get adjacencies from the application and put it in the given list.

auto getDOFMap(std::string &discretization_type)

Get a dof id map from the application.

auto getField(const std::string &field_name)

Get a field with a given name from the application.

void pullField(const std::string &field_name, Field<Scalar, Kokkos::LayoutLeft, MemorySpace> field)

Pull a field with a given name to the application.

void pushField(const std::string &field_name, const Field<Scalar, Kokkos::LayoutLeft, MemorySpace> field)

Push a field with a given name to the application.

void evaluateField(const std::string &field_name, const EvaluationSet<Kokkos::LayoutLeft, MemorySpace> eval_set, Field<Scalar, Kokkos::LayoutLeft, MemorySpace> field)

Ask the application to evaluate a field with a given name.

C API

C interface header.

Typedefs

typedef struct _DTK_UserApplicationHandle *DTK_UserApplicationHandle

DTK user application handle.

Must be created using DTK_createUserApplication() to be a valid handle.

The handle essentially hides C++ implementation details from the user.

typedef struct _DTK_MapHandle *DTK_MapHandle

DTK map handle.

Must be created using DTK_createMap() to be a valid handle.

The handle essentially hides C++ implementation details from the user.

typedef void( * DTK_NodeListSizeFunction) (void *user_data, unsigned *space_dim, size_t *local_num_nodes)

Prototype function to get the size parameters for building a node list.

Register with a user application using DTK_setUserFunction() by passing DTK_NODE_LIST_SIZE_FUNCTION as the type argument.

Parameters
  • user_data: Pointer to custom user data.
  • space_dim: Spatial dimension.
  • local_num_nodes: Number of nodes DTK will allocate memory for.

typedef void( * DTK_NodeListDataFunction) (void *user_data, Coordinate *coordinates)

Prototype function to get the data for a node list.

Register with a user application using DTK_setUserFunction() by passing DTK_NODE_LIST_DATA_FUNCTION as the type argument.

Parameters
  • user_data: Pointer to custom user data.
  • coordinates: Node coordinates.

typedef void( * DTK_BoundingVolumeListSizeFunction) (void *user_data, unsigned *space_dim, size_t *local_num_volumes)

Prototype function to get the size parameters for building a bounding volume list.

Register with a user application using DTK_setUserFunction() by passing DTK_BOUNDING_VOLUME_LIST_SIZE_FUNCTION as the type argument.

Parameters
  • user_data: Pointer to custom user data.
  • space_dim: Spatial dimension.
  • local_num_volumes: Number of volumes DTK will allocate memory for.

typedef void( * DTK_BoundingVolumeListDataFunction) (void *user_data, Coordinate *bounding_volumes)

Prototype function to get the data for a bounding volume list.

Register with a user application using DTK_setUserFunction() by passing DTK_BOUNDING_VOLUME_LIST_DATA_FUNCTION as the type argument.

Parameters
  • user_data: Pointer to custom user data.
  • bounding_volumes: Bounding volumes.

typedef void( * DTK_PolyhedronListSizeFunction) (void *user_data, unsigned *space_dim, size_t *local_num_nodes, size_t *local_num_faces, size_t *total_face_nodes, size_t *local_num_cells, size_t *total_cell_faces)

Prototype function to get the size parameters for building a polyhedron list.

Register with a user application using DTK_setUserFunction() by passing DTK_POLYHEDRON_LIST_SIZE_FUNCTION as the type argument.

Parameters
  • user_data: Pointer to custom user data.
  • space_dim: Spatial dimension.
  • local_num_nodes: Number of nodes DTK will allocate memory for.
  • local_num_faces: Number of faces DTK will allocate memory for.
  • total_face_nodes: Total number of nodes for all faces.
  • local_num_cells: Number of cells DTK will allocate memory for.
  • total_cell_faces: Total number of faces for all cells.

typedef void( * DTK_PolyhedronListDataFunction) (void *user_data, Coordinate *coordinates, LocalOrdinal *faces, unsigned *nodes_per_face, LocalOrdinal *cells, unsigned *faces_per_cell, int *face_orientation)

Prototype function to get the data for a polyhedron list.

Register with a user application using DTK_setUserFunction() by passing DTK_POLYHEDRON_LIST_DATA_FUNCTION as the type argument.

Parameters
  • user_data: Pointer to custom user data.
  • coordinates: Node coordinates.
  • faces: Connectivity list of faces.
  • nodes_per_face: Number of nodes per face.
  • cells: Connectivity list of polyhedrons.
  • faces_per_cell: Number of faces per cell.
  • face_orientation: Orientation of the faces.

typedef void( * DTK_CellListSizeFunction) (void *user_data, unsigned *space_dim, size_t *local_num_nodes, size_t *local_num_cells, size_t *total_cell_nodes)

Prototype function to get the size parameters for building a cell list.

Register with a user application using DTK_setUserFunction() by passing DTK_CELL_LIST_SIZE_FUNCTION as the type argument.

Parameters
  • user_data: Pointer to custom user data.
  • space_dim: Spatial dimension.
  • local_num_nodes: Number of nodes DTK will allocate memory for.
  • local_num_cells: Number of cells DTK will allocate memory for.
  • total_cell_nodes: Total number of nodes for all cells.

typedef void( * DTK_CellListDataFunction) (void *user_data, Coordinate *coordinates, LocalOrdinal *cells, DTK_CellTopology *cell_topologies)

Prototype function to get the data for a mixed topology cell list.

Register with a user application using DTK_setUserFunction() by passing DTK_CELL_LIST_DATA_FUNCTION as the type argument.

Parameters
  • user_data: Pointer to custom user data.
  • coordinates: Node coordinates.
  • cells: List of cells.
  • cell_topologies: Topologies of the cells.

typedef void( * DTK_BoundarySizeFunction) (void *user_data, size_t *local_num_faces)

Prototype function to get the size parameters for a boundary.

Register with a user application using DTK_setUserFunction() by passing DTK_BOUNDARY_SIZE_FUNCTION as the type argument.

Parameters
  • user_data: Pointer to custom user data.
  • local_num_faces: Number of faces owned by this process.

typedef void( * DTK_BoundaryDataFunction) (void *user_data, LocalOrdinal *boundary_cells, unsigned *cell_faces_on_boundary)

Prototype function to get the data for a boundary.

Register with a user application using DTK_setUserFunction() by passing DTK_BOUNDARY_DATA_FUNCTION as the type argument.

Parameters
  • user_data: Pointer to custom user data.
  • boundary_cells: Indices of the cells on the boundary.
  • cell_faces_on_boundary: Indices of the faces within a given cell that is on the boundary.

typedef void( * DTK_AdjacencyListSizeFunction) (void *user_data, size_t *total_adjacencies)

Prototype function to get the size parameters for building an adjacency list.

Register with a user application using DTK_setUserFunction() by passing DTK_ADJACENCY_LIST_SIZE_FUNCTION as the type argument.

Parameters
  • user_data: Pointer to custom user data.
  • total_adjacencies: Total number of adjacencies in the list.

typedef void( * DTK_AdjacencyListDataFunction) (void *user_data, GlobalOrdinal *global_cell_ids, GlobalOrdinal *adjacent_global_cell_ids, unsigned *adjacencies_per_cell)

Prototype function to get the data for an adjacency list.

Register with a user application using DTK_setUserFunction() by passing DTK_ADJACENCY_LIST_DATA_FUNCTION as the type argument.

Parameters
  • user_data: Pointer to custom user data.
  • global_cell_ids: The global ids of the local cells in the list.
  • adjacent_global_cell_ids: The global ids of the cells adjacent to the local cells in the list. These may live on another process.
  • adjacencies_per_cell: The number of adjacencies each local cell has. These serve as offsets into the adjacent_global_cell_ids array.

typedef void( * DTK_DOFMapSizeFunction) (void *user_data, size_t *local_num_dofs, size_t *local_num_objects, unsigned *dofs_per_object)

Prototype function to get the size parameters for a degree-of-freedom id map with a single number of dofs per object.

Register with a user application using DTK_setUserFunction() by passing DTK_DOF_MAP_SIZE_FUNCTION as the type argument.

Parameters
  • user_data: Pointer to custom user data.
  • local_num_dofs: Number of degrees of freedom owned by this process.
  • local_num_objects: Number of objects on this process.
  • dofs_per_objects: Degrees of freedom per object.

typedef void( * DTK_DOFMapDataFunction) (void *user_data, GlobalOrdinal *global_dof_ids, LocalOrdinal *object_dof_ids, char *discretization_type)

Prototype function to get the size data for a degree-of-freedom id map with a single number of dofs per object.

Register with a user application using DTK_setUserFunction() by passing DTK_DOF_MAP_DATA_FUNCTION as the type argument.

Parameters
  • user_data: Pointer to custom user data.
  • global_dof_ids: Globally unique ids for DOFs on this process.
  • object_dof_ids: For every object of the given type in the object list give the local dof ids for that object. The local dof ids correspond to the index of the entry in the global dof id view.
  • discretization_type: Type of discretization.

typedef void( * DTK_MixedTopologyDofMapSizeFunction) (void *user_data, size_t *local_num_dofs, size_t *local_num_objects, size_t *total_dofs_per_object)

Prototype function to get the size parameters for a degree-of-freedom id map with each object having a potentially different number of dofs (e.g. mixed topology cell lists or polyhedron lists).

Register with a user application using DTK_setUserFunction() by passing DTK_MIXED_TOPOLOGY_DOF_MAP_SIZE_FUNCTION as the type argument.

Parameters
  • user_data: Pointer to custom user data.
  • local_num_dofs: Number of degrees of freedom owned by this process.
  • local_num_objects: Number of objects on this process.
  • total_dofs_per_objects: Total degrees of freedom per objects.

typedef void( * DTK_MixedTopologyDofMapDataFunction) (void *user_data, GlobalOrdinal *global_dof_ids, LocalOrdinal *object_dof_ids, unsigned *dofs_per_object, char *discretization_type)

Prototype function to get the data for a multiple object degree-of-freedom id map (e.g. mixed topology cell lists or polyhedron lists).

Register with a user application using DTK_setUserFunction() by passing DTK_MIXED_TOPOLOGY_DOF_MAP_DATA_FUNCTION as the type argument.

Parameters
  • user_data: Pointer to custom user data.
  • global_dof_ids: Globally unique ids for DOFs on this process.
  • object_dof_ids: Local object IDs.
  • dofs_per_object: Degrees of freedom per object.
  • discretization_type: Type of discretization.

typedef void( * DTK_FieldSizeFunction) (void *user_data, const char *field_name, unsigned *field_dimension, size_t *local_num_dofs)

Prototype function to get the size parameters for a field.

Register with a user application using DTK_setUserFunction() by passing DTK_FIELD_SIZE_FUNCTION as the type argument.

Field must be of size local_num_dofs in the associated dof_id_map.

Parameters
  • user_data: Custom user data.
  • field_name: Name of the field.
  • field_dimension: Dimension of the field (i.e. 1 for the pressure, or 3 for the velocity in 3-D)
  • local_num_dofs: Number of degrees of freedom owned by this process.

typedef void( * DTK_PullFieldDataFunction) (void *user_data, const char *field_name, double *field_dofs)

Prototype function to pull data from the application into a field.

Register with a user application using DTK_setUserFunction() by passing DTK_PULL_FIELD_DATA_FUNCTION as the type argument.

Parameters
  • user_data: Custom user data.
  • field_name: Name of the field to pull.
  • field_dofs: Degrees of freedom for that field.

typedef void( * DTK_PushFieldDataFunction) (void *user_data, const char *field_name, const double *field_dofs)

Prototype function to push data from a field into the application.

Register with a user application using DTK_setUserFunction() by passing DTK_PUSH_FIELD_DATA_FUNCTION as the type argument.

Parameters
  • user_data: Custom user data.
  • field_name: Name of the field to push.
  • field_dofs: Degrees of freedom for that field.

typedef void( * DTK_EvaluateFieldFunction) (void *user_data, const char *field_name, const Coordinate *evaluation_points, const LocalOrdinal *object_ids, double *values)

Prototype function to evaluate a field at a given set of points in a given set of objects.

Register with a user application using DTK_setUserFunction() by passing DTK_EVALUATE_FIELD_FUNCTION as the type argument.

Parameters
  • user_data: Custom user data.
  • field_name: Name of the field to evaluate.
  • evaluate_points: Coordinates of the points at which to evaluate the field.
  • objects_ids: ID of the cell/face with repect of which the coordinates are expressed.
  • values: Field values.

Enums

enum DTK_MemorySpace

Memory space (where memory is allocated)

Values:

DTK_HOST_SPACE
DTK_CUDAUVM_SPACE
enum DTK_ExecutionSpace

Execution space (where functions execute)

Values:

DTK_SERIAL
DTK_OPENMP
DTK_CUDA
enum DTK_Error

DTK error codes.

Values:

DTK_SUCCESS = 0
DTK_INVALID_HANDLE = -1
DTK_UNINITIALIZED = -2
DTK_UNKNOWN = -99
enum DTK_FunctionType

Passed as the type argument to DTK_setUserFunction() in order to indicate what callback function is being registered with the user application.

Note
Callback functions are passed as pointers to functions that take no arguments and return nothing (void(*)()) so the value of the DTK_FunctionType enum is necessary to indicate what is being registered with the user application and how to cast the function pointer back to the appropriate signature.

Values:

DTK_NODE_LIST_SIZE_FUNCTION

See DTK_NodeListSizeFunction()

DTK_NODE_LIST_DATA_FUNCTION

See DTK_NodeListDataFunction()

DTK_BOUNDING_VOLUME_LIST_SIZE_FUNCTION

See DTK_BoundingVolumeListSizeFunction()

DTK_BOUNDING_VOLUME_LIST_DATA_FUNCTION

See DTK_BoundingVolumeListDataFunction()

DTK_POLYHEDRON_LIST_SIZE_FUNCTION

See DTK_PolyhedronListSizeFunction()

DTK_POLYHEDRON_LIST_DATA_FUNCTION

See DTK_PolyhedronListDataFunction()

DTK_CELL_LIST_SIZE_FUNCTION

See DTK_CellListSizeFunction()

DTK_CELL_LIST_DATA_FUNCTION

See DTK_CellListDataFunction()

DTK_BOUNDARY_SIZE_FUNCTION

See DTK_BoundarySizeFunction()

DTK_BOUNDARY_DATA_FUNCTION

See DTK_BoundaryDataFunction()

DTK_ADJACENCY_LIST_SIZE_FUNCTION

See DTK_AdjacencyListSizeFunction()

DTK_ADJACENCY_LIST_DATA_FUNCTION

See DTK_AdjacencyListDataFunction()

DTK_DOF_MAP_SIZE_FUNCTION

See DTK_DOFMapSizeFunction()

DTK_DOF_MAP_DATA_FUNCTION

See DTK_DOFMapDataFunction()

DTK_MIXED_TOPOLOGY_DOF_MAP_SIZE_FUNCTION

See DTK_MixedTopologyDofMapSizeFunction()

DTK_MIXED_TOPOLOGY_DOF_MAP_DATA_FUNCTION

See DTK_MixedTopologyDofMapDataFunction()

DTK_FIELD_SIZE_FUNCTION

See DTK_FieldSizeFunction()

DTK_PULL_FIELD_DATA_FUNCTION

See DTK_PullFieldDataFunction()

DTK_PUSH_FIELD_DATA_FUNCTION

See DTK_PushFieldDataFunction()

DTK_EVALUATE_FIELD_FUNCTION

See DTK_EvaluateFieldFunction()

Functions

const char *DTK_version()

Get the current version of DTK.

Return
Returns a string containing the version number for DTK.

const char *DTK_git_commit_hash()

Get the current repository hash.

Note
If the source code is not under revision control (e.g. downloaded as a tarball), this functions returns an error string indicating that it is not a GIT repository.
Return
Returns a string containing the revision number.

DTK_UserApplicationHandle DTK_createUserApplication(DTK_MemorySpace space)

Create a DTK handle to a user application.

Return
DTK_create returns a handle for the user application.
Parameters
  • space: Execution space for the callback functions that are to be registered using DTK_setUserFunction().

bool DTK_isValidUserApplication(DTK_UserApplicationHandle handle)

Indicates whether a DTK handle to a user application is valid.

A handle is valid if it was created by DTK_create() and has not yet been deleted by DTK_destroyUserApplication().

Return
true if the given user application handle is valid; false otherwise.
Parameters
  • handle: The DTK user application handle to check.

void DTK_destroyUserApplication(DTK_UserApplicationHandle handle)

Destroy a DTK handle to a user application.

Parameters
  • handle: User application handle.

DTK_MapHandle DTK_createMap(DTK_ExecutionSpace space, MPI_Comm comm, DTK_UserApplicationHandle source, DTK_UserApplicationHandle target)

Create a DTK handle to a user appliction.

Return
DTK_create returns a handle for the map.
Parameters
  • space: Execution space where the map will execute.
  • comm: The MPI communicator over which to build the map.
  • source: Handle to the source application.
  • target: Handle to the target application.

bool DTK_isValidMap(DTK_MapHandle handle)

Indicates whether a DTK handle to a map is valid.

A handle is valid if it was created by DTK_create() and has not yet been deleted by DTK_destroyUserApplication().

Return
true if the given map handle is valid; false otherwise.
Parameters
  • handle: The DTK map handle to check.

void DTK_applyMap(DTK_MapHandle handle, const char *source_field, const char *target_field)

Apply the DTK map to the given fields.

Parameters
  • handle: Map handle.
  • source_field: Name of the field in the source application.
  • target_field: Name of the field in the target application.

void DTK_destroyMap(DTK_MapHandle handle)

Destroy a DTK handle to a map.

Parameters
  • handle: map handle.

void DTK_initialize()

Initializes the DTK execution environment.

This initializes Kokkos if it has not already been initialized.

void DTK_initialize_cmd(int *argc, char ***argv)

Initialize DTK.

This initializes Kokkos if it has not already been initialized.

This version of initialize() effectively calls Kokkos::initialize( *argc, *argv ). Pointers to argc and argv arguments are passed in order to match MPI_Init’s interface. This function name was suffixed with _cmd because, unlike C++, C does not allow to overload functions.

Parameters
  • argc: Pointer to the number of argument.
  • argv: Pointer to the argument vector.

bool DTK_is_initialized()

Indicates whether DTK has been initialized.

This function may be used to determine whether DTK has been initialized.

void DTK_finalize()

Finalizes DTK.

This function terminates the DTK execution environment. If DTK initialized Kokkos, finalize Kokkos. However, if Kokkos was initialized before DTK, then this function does NOT finalize Kokkos.

const char *DTK_error(int err)

Get DTK error message.

All DTK functions set errno error code upon completion. If DTK function fails, the code is nonzero. This function provides a way to get the the error message associated with the error code. If the error code is unknown, DTK returns a string stating that.

Return
Returns corresponding error string. If the error code is 0 (success), return empty string.
Parameters
  • err: Error number (typically, errno)

void DTK_setUserFunction(DTK_UserApplicationHandle handle, DTK_FunctionType type, void (*f)()void *user_data, )

Register a function as a callback.

This registers a custom function as a callback for DTK to communicate with the user application.

Parameters
  • handle: User application handle.
  • type: Type of callback function.
  • f: Pointer to user defined callback function.
  • user_data: Pointer to the user data that will be passed to the callback function when executing it.