Exceptions to the PMIx Standard
===============================

.. |br| raw:: html

    <br/>

Exceptions to the base PMIx Standard are listed here. These exceptions
are not indicative of any intent to stray
from the Standard, but instead represent the difference between the
pace of development of the library versus the normal Standard's
process. Accordingly, it is expected that the exceptions listed below
will make their way into a future release of the PMIx Standard and
then be removed from the list of exceptions in some future OpenPMIx
release.

Extensions
----------

OpenPMIx |opmix_ver| is based on the PMIx |std_ver| Standard. In
addition to *Extensions* to the Standard, this release includes the conversion of
all support macros to PMIx function APIs |mdash| e.g., the
``PMIX_LOAD_PROCID`` macro is now the ``PMIx_Load_procid()``
function |mdash| in accordance with planned changes to the Standard.
The macro versions have been retained as deprecated (without
warnings) for backward compatibility.

Utility APIs
------------

* Load a key:

  .. code-block:: c

     void PMIx_Load_key(pmix_key_t key, const char *src);

* Check a key:

  .. code-block:: c

     bool PMIx_Check_key(const char *key, const char *str);

* Check to see if a key is a "reserved" key:

  .. code-block:: c

     bool PMIx_Check_reserved_key(const char *key);

* Load a string into a ``pmix_nspace_t`` struct:

  .. code-block:: c

     void PMIx_Load_nspace(pmix_nspace_t nspace, const char *str);

* Check two ``nspace`` structs for equality:

  .. code-block:: c

     bool PMIx_Check_nspace(const char *key1, const char *key2);

* Check if a namespace is invalid:

  .. code-block:: c

     bool PMIx_Nspace_invalid(const char *nspace);

* Load a process ID struct:

  .. code-block:: c

     void PMIx_Load_procid(pmix_proc_t *p,
                           const char *ns,
                           pmix_rank_t rk);

* Transfer a process ID struct (non-destructive):

  .. code-block:: c

     void PMIx_Xfer_procid(pmix_proc_t *dst,
                           const pmix_proc_t *src);

* Check two proc IDs for equality:

  .. code-block:: c

     bool PMIx_Check_procid(const pmix_proc_t *a,
                            const pmix_proc_t *b);

* Check two ranks for equality:

  .. code-block:: c

     bool PMIx_Check_rank(pmix_rank_t a,
                          pmix_rank_t b);

* Check if proc ID is invalid:

  .. code-block:: c

     bool PMIx_Procid_invalid(const pmix_proc_t *p);

* Check if the tool is connected to a PMIx server:

  .. code-block:: c

     bool PMIx_tool_is_connected(void);


Argv Handling
-------------
Functions for handling of argv arrays (``NULL``-terminated array of strings)

* Count the number of entries

.. code-block:: c

    int PMIx_Argv_count(char **a);

* Append a string to the array

.. code-block:: c

    pmix_status_t PMIx_Argv_append_nosize(char ***argv, const char *arg);

* Prepend a string to the array

.. code-block:: c

    pmix_status_t PMIx_Argv_prepend_nosize(char ***argv, const char *arg);

* Append a string to the array, but only if it doesn't already
  appear on the array (ignore if it does)

.. code-block:: c

    pmix_status_t PMIx_Argv_append_unique_nosize(char ***argv, const char *arg);

* Free an array, including each string on the array

.. code-block:: c

    void PMIx_Argv_free(char **argv);

* Split a string into an argv array, dividing the string on each occurrence
  of the specified delimiter character. Retain empty entries in the array
  when more than one copy of the delimiter character appears in a sequence.

.. code-block:: c

     char **PMIx_Argv_split_inter(const char *src_string,
                                  int delimiter,
                                  bool include_empty);

* Split a string into an argv array, dividing the string on each occurrence
  of the specified delimiter character. Retain empty entries in the array
  when more than one copy of the delimiter character appears in a sequence.
  Acts as a wrapper to ``PMIx_Argv_split_inter`` with ``include_empty`` set
  to ``true``

.. code-block:: c

    char **PMIx_Argv_split_with_empty(const char *src_string, int delimiter);

* Split a string into an argv array, dividing the string on each occurrence
  of the specified delimiter character. Discard empty entries in the array
  when more than one copy of the delimiter character appears in a sequence.
  Acts as a wrapper to ``PMIx_Argv_split_inter`` with ``include_empty`` set
  to ``false``

.. code-block:: c

    char **PMIx_Argv_split(const char *src_string, int delimiter);

* Join all the elements of an argv array into a single newly-allocated string,
  with the specified delimiter character at the join points.

.. code-block:: c

    char *PMIx_Argv_join(char **argv, int delimiter);

* Copy a ``NULL``-terminated argv array.

.. code-block:: c

    char **PMIx_Argv_copy(char **argv);

* Set environment variable:

  .. code-block:: c

     pmix_status_t PMIx_Setenv(const char *name,
                               const char *value,
                               bool overwrite,
                               char ***env);

Value Struct Functions
----------------------
* Initialize a value struct:

  .. code-block:: c

     void PMIx_Value_construct(pmix_value_t *val);

* Free memory stored inside a value struct:

  .. code-block:: c

     void PMIx_Value_destruct(pmix_value_t *val);

* Create and initialize an array of value structs:

  .. code-block:: c

     pmix_value_t* PMIx_Value_create(size_t n);

* Free memory stored inside an array of coord structs (does
  not free the struct memory itself):

  .. code-block:: c

     void PMIx_Value_free(pmix_value_t *v, size_t n);

* Check the given value struct to determine if it includes a boolean
  value (includes strings for ``true`` and ``false``, including
  abbreviations such as ``t`` or ``f``), and if so, then its value. A
  value type of ``PMIX_UNDEF`` is taken to imply a boolean ``true``.

  .. code-block:: c

     pmix_boolean_t PMIx_Value_true(const pmix_value_t *v);

* Compare the contents of two ``pmix_value_t`` structures:

  .. code-block:: c

     pmix_value_cmp_t PMIx_Value_compare(pmix_value_t *v1,
                                         pmix_value_t *v2);

* Get the size of the contents of a ``pmix_value_t`` structure:

  .. code-block:: c

     pmix_status_t PMIx_Value_get_size(const pmix_value_t *val,
                                       size_t *size);

Data Array Functions
--------------------
* Construct a data array object, allocating the memory for the indicated
  number of the specified data type. Memory for the provided data array
  object must have previously been allocated or statically declared:

  .. code-block:: c

     void PMIx_Data_array_construct(pmix_data_array_t *p,
                                    size_t num, pmix_data_type_t type);

* Initialize the fields of a data array object without allocating any
  memory for the included array:

  .. code-block:: c

     void PMIx_Data_array_init(pmix_data_array_t *p,
                               pmix_data_type_t type);

* Destroy a data array object, releasing all memory included in it:

  .. code-block:: c

     void PMIx_Data_array_destruct(pmix_data_array_t *d);

* Create and initialize a ``pmix_data_array_t`` structure, allocating the
  memory for the indicated number of the specified data type as well as
  the ``pmix_data_array_t`` object itself:

  .. code-block:: c

     pmix_data_array_t* PMIx_Data_array_create(size_t n, pmix_data_type_t type);

* Free memory stored inside a ``pmix_data_array_t`` structure (does not free
  the provided ``pmix_data_array_t`` object itself):

  .. code-block:: c

     void PMIx_Data_array_free(pmix_data_array_t *p);

Info Struct Functions
---------------------
* Initialize an info struct. Memory for the provided
  object must have previously been allocated or statically declared:

  .. code-block:: c

     void PMIx_Info_construct(pmix_info_t *p);

* Free memory stored inside an info struct:

  .. code-block:: c

     void PMIx_Info_destruct(pmix_info_t *p);

* Create and initialize an array of info structs:

  .. code-block:: c

     pmix_info_t* PMIx_Info_create(size_t n);

* Free memory stored inside an array of coord structs (does
  not free the struct memory itself):

  .. code-block:: c

     void PMIx_Info_free(pmix_info_t *p, size_t n);

* Check the given info struct to determine if it includes
  a boolean value (includes strings for ``true`` and ``false``,
  including abbreviations such as ``t`` or ``f``), and if so,
  then its value. A value type of ``PMIX_UNDEF`` is taken to imply
  a boolean ``true`` as the presence of the key defaults to
  indicating ``true``.

  .. code-block:: c

     pmix_boolean_t PMIx_Info_true(const pmix_info_t *p);

* Mark the info struct as required:

  .. code-block:: c

     void PMIx_Info_required(pmix_info_t *p);

* Mark the info struct as optional:

  .. code-block:: c

     void PMIx_Info_optional(pmix_info_t *p);

* Check if the info struct is required:

  .. code-block:: c

     bool PMIx_Info_is_required(const pmix_info_t *p);

* Check if the info struct is optional:

  .. code-block:: c

     bool PMIx_Info_is_optional(const pmix_info_t *p);

* Mark the info struct as processed:

  .. code-block:: c

     void PMIx_Info_processed(pmix_info_t *p);

* Check if the info struct has been processed:

  .. code-block:: c

     bool PMIx_Info_was_processed(const pmix_info_t *p);

* Mark the info struct as the end of an array:

  .. code-block:: c

     void PMIx_Info_set_end(pmix_info_t *p);

* Check if the info struct is the end of an array:

  .. code-block:: c

     bool PMIx_Info_is_end(const pmix_info_t *p);

* Mark the info as a qualifier:

  .. code-block:: c

     void PMIx_Info_qualifier(pmix_info_t *p);

* Check if the info struct is a qualifier:

  .. code-block:: c

     bool PMIx_Info_is_qualifier(const pmix_info_t *p);

* Mark the info struct as persistent |mdash| do *not* release its contents:

  .. code-block:: c

     void PMIx_Info_persistent(pmix_info_t *p);

* Check if the info struct is persistent:

  .. code-block:: c

     bool PMIx_Info_is_persistent(const pmix_info_t *p);

* Get the size of a ``pmix_info_t`` structure:

  .. code-block:: c

      pmix_status_t PMIx_Info_get_size(const pmix_info_t *val,
                                       size_t *size);

Coordinate Struct Functions
---------------------------
* Initialize a coord struct. Memory for the provided
  object must have previously been allocated or statically declared:

  .. code-block:: c

     void PMIx_Coord_construct(pmix_coord_t *m);

* Free memory stored inside a coord struct:

  .. code-block:: c

     void PMIx_Coord_destruct(pmix_coord_t *m);

* Create and initialize an array of coord structs:

  .. code-block:: c

     pmix_coord_t* PMIx_Coord_create(size_t dims,
                                     size_t number);

* Free memory stored inside an array of coord structs (does
  not free the struct memory itself):

  .. code-block:: c

     void PMIx_Coord_free(pmix_coord_t *m, size_t number);

Topology Functions
------------------
* Initialize a topology struct. Memory for the provided
  object must have previously been allocated or statically declared:

  .. code-block:: c

     void PMIx_Topology_construct(pmix_topology_t *t);

* Create and initialize an array of topology structs:

  .. code-block:: c

     pmix_topology_t* PMIx_Topology_create(size_t n);

* Free memory stored inside an array of topology structs (does
  not free the struct memory itself):

  .. code-block:: c

     void PMIx_Topology_free(pmix_topology_t *t, size_t n);

Cpuset Functions
----------------
* Initialize a cpuset struct. Memory for the provided
  object must have previously been allocated or statically declared:

  .. code-block:: c

     void PMIx_Cpuset_construct(pmix_cpuset_t *cpuset);

* Free memory stored inside a cpuset struct:

  .. code-block:: c

     void PMIx_Cpuset_destruct(pmix_cpuset_t *cpuset);

* Create and initialize an array of cpuset structs:

  .. code-block:: c

     pmix_cpuset_t* PMIx_Cpuset_create(size_t n);

* Free memory stored inside an array of cpuset structs (does
  not free the struct memory itself):

  .. code-block:: c

     void PMIx_Cpuset_free(pmix_cpuset_t *c, size_t n);

Geometry Functions
------------------
* Initialize a geometry struct. Memory for the provided
  object must have previously been allocated or statically declared:

  .. code-block:: c

     void PMIx_Geometry_construct(pmix_geometry_t *g);

* Free memory stored inside a cpuset struct:

  .. code-block:: c

     void PMIx_Geometry_destruct(pmix_geometry_t *g);

* Create and initialize an array of cpuset structs:

  .. code-block:: c

     pmix_geometry_t* PMIx_Geometry_create(size_t n);

* Free memory stored inside an array of cpuset structs (does
  not free the struct memory itself):

  .. code-block:: c

     void PMIx_Geometry_free(pmix_geometry_t *g, size_t n);

Device Distance Functions
-------------------------
* Initialize a device distance struct. Memory for the provided
  object must have previously been allocated or statically declared:

  .. code-block:: c

     void PMIx_Device_distance_construct(pmix_device_distance_t *d);

* Free memory stored inside a device distance struct:

  .. code-block:: c

     void PMIx_Device_distance_destruct(pmix_device_distance_t *d);

* Create and initialize an array of device distance structs:

  .. code-block:: c

     pmix_device_distance_t* PMIx_Device_distance_create(size_t n);

* Free memory stored inside an array of device distance structs (does
  not free the struct memory itself):

  .. code-block:: c

     void PMIx_Device_distance_free(pmix_device_distance_t *d, size_t n);

Byte Object Functions
---------------------
* Initialize a byte object struct. Memory for the provided
  object must have previously been allocated or statically declared:

  .. code-block:: c

     void PMIx_Byte_object_construct(pmix_byte_object_t *b);

* Free memory stored inside a byte object struct:

  .. code-block:: c

     void PMIx_Byte_object_destruct(pmix_byte_object_t *g);

* Create and initialize an array of byte object structs:

  .. code-block:: c

     pmix_byte_object_t* PMIx_Byte_object_create(size_t n);

* Free memory stored inside an array of byte object structs (does
  not free the struct memory itself):

  .. code-block:: c

     void PMIx_Byte_object_free(pmix_byte_object_t *g, size_t n);

* Load a byte object:

  .. code-block:: c

     void PMIx_Byte_object_load(pmix_byte_object_t *b,
                                char *d, size_t sz);

Endpoint Functions
------------------
* Initialize an endpoint struct. Memory for the provided
  object must have previously been allocated or statically declared:

  .. code-block:: c

     void PMIx_Endpoint_construct(pmix_endpoint_t *e);

* Free memory stored inside an endpoint struct:

  .. code-block:: c

     void PMIx_Endpoint_destruct(pmix_endpoint_t *e);

* Create and initialize an array of endpoint structs:

  .. code-block:: c

     pmix_endpoint_t* PMIx_Endpoint_create(size_t n);

* Free memory stored inside an array of endpoint structs (does
  not free the struct memory itself):

  .. code-block:: c

     void PMIx_Endpoint_free(pmix_endpoint_t *e, size_t n);

Envar Functions
---------------
* Initialize an envar struct. Memory for the provided
  object must have previously been allocated or statically declared:

  .. code-block:: c

     void PMIx_Envar_construct(pmix_envar_t *e);

* Free memory stored inside an envar struct:

  .. code-block:: c

     void PMIx_Envar_destruct(pmix_envar_t *e);

* Create and initialize an array of envar structs:

  .. code-block:: c

     pmix_envar_t* PMIx_Envar_create(size_t n);

* Free memory stored inside an array of envar structs (does
  not free the struct memory itself):

  .. code-block:: c

     void PMIx_Envar_free(pmix_envar_t *e, size_t n);

* Load an envar struct:

  .. code-block:: c

     void PMIx_Envar_load(pmix_envar_t *e,
                          char *var,
                          char *value,
                          char separator);

Data Buffer Functions
---------------------
* Initialize a data buffer struct. Memory for the provided
  object must have previously been allocated or statically declared:

  .. code-block:: c

     void PMIx_Data_buffer_construct(pmix_data_buffer_t *b);

* Free memory stored inside a data buffer struct:

  .. code-block:: c

     void PMIx_Data_buffer_destruct(pmix_data_buffer_t *b);

* Create a data buffer struct:

  .. code-block:: c

     pmix_data_buffer_t* PMIx_Data_buffer_create(void);

* Free memory stored inside a data buffer struct:

  .. code-block:: c

     void PMIx_Data_buffer_release(pmix_data_buffer_t *b);

* Load a data buffer struct:

  .. code-block:: c

     void PMIx_Data_buffer_load(pmix_data_buffer_t *b,
                                char *bytes, size_t sz);

* Unload a data buffer struct:

  .. code-block:: c

     void PMIx_Data_buffer_unload(pmix_data_buffer_t *b,
                                  char **bytes, size_t *sz);

Proc Struct Functions
---------------------
* Initialize a proc struct. Memory for the provided
  object must have previously been allocated or statically declared:

  .. code-block:: c

     void PMIx_Proc_construct(pmix_proc_t *p);

* Clear memory inside a proc struct:

  .. code-block:: c

     void PMIx_Proc_destruct(pmix_proc_t *p);

* Create and initialize an array of proc structs:

  .. code-block:: c

     pmix_proc_t* PMIx_Proc_create(size_t n);

* Free memory stored inside an array of proc structs (does
  not free the struct memory itself):

  .. code-block:: c

     void PMIx_Proc_free(pmix_proc_t *p, size_t n);

* Load a proc struct:

  .. code-block:: c

     void PMIx_Proc_load(pmix_proc_t *p,
                         char *nspace, pmix_rank_t rank);

* Construct a multicluster ``nspace`` struct from cluster and
  ``nspace`` values:

  .. code-block:: c

     void PMIx_Multicluster_nspace_construct(pmix_nspace_t target,
                                             pmix_nspace_t cluster,
                                             pmix_nspace_t nspace);

* Parse a multicluster nspace struct to separate out the cluster
  and ``nspace`` portions:

  .. code-block:: c

     void PMIx_Multicluster_nspace_parse(pmix_nspace_t target,
                                         pmix_nspace_t cluster,
                                         pmix_nspace_t nspace);

Proc Info Functions
-------------------
* Initialize a proc info struct. Memory for the provided
  object must have previously been allocated or statically declared:

  .. code-block:: c

     void PMIx_Proc_info_construct(pmix_proc_info_t *p);

* Clear memory inside a proc info struct:

  .. code-block:: c

     void PMIx_Proc_info_destruct(pmix_proc_info_t *p);

* Create and initialize an array of proc info structs:

  .. code-block:: c

     pmix_proc_info_t* PMIx_Proc_info_create(size_t n);

* Free memory stored inside an array of proc info structs (does
  not free the struct memory itself):

  .. code-block:: c

     void PMIx_Proc_info_free(pmix_proc_info_t *p, size_t n);

Proc Stats Functions
--------------------
* Initialize a proc stats struct. Memory for the provided
  object must have previously been allocated or statically declared:

  .. code-block:: c

     void PMIx_Proc_stats_construct(pmix_proc_stats_t *p);

* Clear memory inside a proc stats struct:

  .. code-block:: c

     void PMIx_Proc_stats_destruct(pmix_proc_stats_t *p);

* Create and initialize an array of proc stats structs:

  .. code-block:: c

     pmix_proc_stats_t* PMIx_Proc_stats_create(size_t n);

* Free memory stored inside an array of proc stats structs (does
  not free the struct memory itself):

  .. code-block:: c

     void PMIx_Proc_stats_free(pmix_proc_stats_t *p, size_t n);

Disk Stats Functions
--------------------
* Initialize a disk stats struct. Memory for the provided
  object must have previously been allocated or statically declared:

  .. code-block:: c

     void PMIx_Disk_stats_construct(pmix_disk_stats_t *p);

* Clear memory inside a disk stats struct:

  .. code-block:: c

     void PMIx_Disk_stats_destruct(pmix_disk_stats_t *p);

* Create and initialize an array of disk stats structs:

  .. code-block:: c

     pmix_disk_stats_t* PMIx_Disk_stats_create(size_t n);

* Free memory stored inside an array of disk stats structs (does
  not free the struct memory itself):

  .. code-block:: c

     void PMIx_Disk_stats_free(pmix_disk_stats_t *p, size_t n);

Net Stats Functions
-------------------
* Initialize a net stats struct. Memory for the provided
  object must have previously been allocated or statically declared:

  .. code-block:: c

     void PMIx_Net_stats_construct(pmix_net_stats_t *p);

* Clear memory inside a net stats struct:

  .. code-block:: c

     void PMIx_Net_stats_destruct(pmix_net_stats_t *p);

* Create and initialize an array of net stats structs:

  .. code-block:: c

     pmix_net_stats_t* PMIx_Net_stats_create(size_t n);

* Free memory stored inside an array of net stats structs (does
  not free the struct memory itself):

  .. code-block:: c

     void PMIx_Net_stats_free(pmix_net_stats_t *p, size_t n);

Process Data Functions
----------------------
* Initialize a pdata struct. Memory for the provided
  object must have previously been allocated or statically declared:

  .. code-block:: c

     void PMIx_Pdata_construct(pmix_pdata_t *p);

* Clear memory inside a pdata struct:

  .. code-block:: c

     void PMIx_Pdata_destruct(pmix_pdata_t *p);

* Create and initialize an array of pdata structs:

  .. code-block:: c

     pmix_pdata_t* PMIx_Pdata_create(size_t n);

* Free memory stored inside an array of pdata structs (does
  not free the struct memory itself):

  .. code-block:: c

     void PMIx_Pdata_free(pmix_pdata_t *p, size_t n);

App Struct Functions
--------------------
* Initialize a ``pmix_app_t`` struct. Memory for the provided
  object must have previously been allocated or statically declared:

  .. code-block:: c

     void PMIx_App_construct(pmix_app_t *p);

* Clear memory inside an app struct:

  .. code-block:: c

     void PMIx_App_destruct(pmix_app_t *p);

* Create and initialize an array of app structs:

  .. code-block:: c

     pmix_app_t* PMIx_App_create(size_t n);

* Create and initialize an array of ``pmix_info_t`` structs
  in the provided ``pmix_app_t`` object:

  .. code-block:: c

     void PMIx_App_info_create(pmix_app_t *p, size_t n);

* Free memory stored inside an array of app structs (does
  not free the struct memory itself):

  .. code-block:: c

     void PMIx_App_free(pmix_app_t *p, size_t n);

* Free memory stored inside a ``pmix_app_t`` object

  .. code-block:: c

     void PMIx_App_release(pmix_app_t *p);

PMIx Info List Functions
------------------------
Constructing arrays of ``pmix_info_t`` for passing to an API can
be tedious since the ``pmix_info_t`` itself is not a "list object".
Since this is a very frequent operation, a set of APIs has been
provided that opaquely manipulates internal PMIx list structures
for this purpose. The user only need provide a ``void*`` pointer to
act as the caddy for the internal list object. The base functions
for these operations are in the Standard, but the following functions
have been added here:

* Retrieve the next ``pmix_info_t`` from the provided list, given
  the current pointer. Passing a ``NULL`` to the ``prev`` parameter
  will return the first object on the list. A ``NULL`` is returned
  upon reaching the end of the list:

    .. code-block:: c

       pmix_info_t* PMIx_Info_list_get_info(void *ptr, void *prev, void **next);

* Insert a `pmix_info_t`` struct into the provided list. This directly
  copies the contents of the provided ``pmix_info_t`` struct, preserving
  any included pointers. The object on the list is subsequently marked
  as ``persistent`` to avoid free'ing any objects pointed to in the struct:

    .. code-block:: c

       pmix_status_t PMIx_Info_list_insert(void *ptr, pmix_info_t *info);

* Prepend a value onto the provided list:

    .. code-block:: c

       pmix_status_t PMIx_Info_list_prepend(void *ptr,
                                            const char *key,
                                            const void *value,
                                            pmix_data_type_t type);


Pretty-Print Functions
-----------------------
The following pretty-print support APIs have been added:

* Print a ``pmix_value_cmp_t`` value

  .. code-block:: c

     const char* PMIx_Value_comparison_string(pmix_value_cmp_t cmp);

* Print the contents of a ``pmix_app_t`` struct. Note that the returned
  string must be free'd by the caller:

   .. code-block:: c

    char* PMIx_App_string(const pmix_app_t *app);

The following pretty-print support APIs have been slightly modified
to add a ``const`` qualifier to their input parameter:

  .. code-block:: c

     const char* PMIx_Get_attribute_string(const char *attribute);
     const char* PMIx_Get_attribute_name(const char *attrstring);
     char* PMIx_Info_string(const pmix_info_t *info);
     char* PMIx_Value_string(const pmix_value_t *value);

  This is not expected to cause any issues for users.

The following function has been added to return the ``pmix_status_t``
corresponding to the string name of the constant:

   .. code-block:: c

    pmix_status_t PMIx_Error_code(const char *errname);


Constants
---------

* ``PMIX_DATA_BUFFER``: data type for packing/unpacking of
  ``pmix_data_buffer_t`` objects
* ``PMIX_DISK_STATS``: data type for packing/unpacking of
  ``pmix_disk_stats_t`` objects
* ``PMIX_NET_STATS``: data type for packing/unpacking of
  ``pmix_net_stats_t`` objects
* ``PMIX_NODE_STATS``: data type for packing/unpacking of
  ``pmix_node_stats_t`` objects
* ``PMIX_PROC_STATS``: data type for packing/unpacking of
  ``pmix_proc_stats_t`` objects
* ``PMIX_ERR_JOB_EXE_NOT_FOUND``: specified executable not found
* ``PMIX_ERR_JOB_INSUFFICIENT_RESOURCES``: insufficient resources to
  spawn job
* ``PMIX_ERR_JOB_SYS_OP_FAILED``: system library operation failed
* ``PMIX_ERR_JOB_WDIR_NOT_FOUND``: specified working directory not
  found
* ``PMIX_READY_FOR_DEBUG``: event indicating job/proc is ready for
  debug (accompanied by ``PMIX_BREAKPOINT`` indicating where proc is
  waiting)
* ``PMIX_ERR_PROC_REQUESTED_ABORT``: process called ``PMIx_Abort``
* ``PMIX_ERR_PROC_KILLED_BY_CMD``: process was terminated by RTE
  command
* ``PMIX_ERR_PROC_FAILED_TO_START``: process failed to start
* ``PMIX_ERR_PROC_ABORTED_BY_SIG``: process aborted by signal (e.g.,
  segmentation fault)
* ``PMIX_ERR_PROC_SENSOR_BOUND_EXCEEDED``: process terminated due to
  exceeding a sensor boundary
* ``PMIX_ERR_EXIT_NONZERO_TERM``: process exited normally, but with a
  non-zero status
* ``PMIX_INFO_QUALIFIER``  (value: 0x00000008): Info is a qualifier to the primary value
* ``PMIX_INFO_PERSISTENT`` (value: 0x00000010): Do not release included value


.. note:: OpenPMIx version |opmix_ver| renamed the  ``PMIX_DEBUG_WAIT_FOR_NOTIFY``
          to ``PMIX_READY_FOR_DEBUG``. The prior name is retained as deprecated
          for backward compatibility.

Attributes
----------

.. list-table::
   :header-rows: 1

   * - Attribute
     - Type
     - Description

   * - ``PMIX_EXTERNAL_AUX_EVENT_BASE`` |br| ``"pmix.evaux"``
     - ``(void*)``
     - event base to be used for auxiliary |br|
       functions (e.g., capturing signals) that |br| would
       otherwise interfere with the |br|
       host
       
   * - ``PMIX_CONNECT_TO_SCHEDULER`` |br| ``"pmix.cnct.sched"``
     - ``(bool)``
     - Connect to the system scheduler
       
   * - ``PMIX_BIND_PROGRESS_THREAD`` |br| ``"pmix.bind.pt"``
     - ``(char*)``
     - Comma-delimited ranges of CPUs |br|
       that the internal PMIx progress |br|
       thread shall be bound to
         
   * - ``PMIX_BIND_REQUIRED`` |br| ``"pmix.bind.reqd"``
     - ``(bool)``
     - Return error if the internal PMIx |br|
       progress thread cannot be bound
           
   * - ``PMIX_COLOCATE_PROCS`` |br| ``"pmix.colproc"``
     - ``(pmix_data_array_t*)``
     - Array of ``pmix_proc_t`` identifying the |br|
       procs with which the new job's procs |br|
       are to be colocated
       
   * - ``PMIX_COLOCATE_NPERPROC`` |br| ``"pmix.colnum.proc"``
     - ``(uint16_t)``
     - Number of procs to colocate with |br|
       each identified proc
       
   * - ``PMIX_COLOCATE_NPERNODE`` |br| ``"pmix.colnum.node"``
     - ``(uint16_t)``
     - Number of procs to colocate on the |br|
       node of each identified proc
       
   * - ``PMIX_EVENT_ONESHOT`` |br| ``pmix.evone``
     - ``(bool)``
     - when registering, indicate that this |br|
       event handler is to be deleted after |br|
       being invoked

   * - ``PMIX_GROUP_ADD_MEMBERS`` |br| ``pmix.grp.add``
     - ``(pmix_data_array_t*)``
     - Array of ``pmix_proc_t`` identifying |br|
       procs that are not included in the |br|
       membership specified in the procs |br|
       array passed to the |br|
       ``PMIx_Group_construct[_nb]()`` call, |br|
       but are to be included in the final |br|
       group. The identified procs will be |br|
       sent an invitation to join the group |br|
       during the construction procedure. |br|
       This is used when some members of |br|
       the proposed group do not know the |br|
       full membership and therefore cannot |br|
       include all members in the call to |br|
       construct.
       
   * - ``PMIX_GROUP_LOCAL_CID`` |br| ``pmix.grp.lclid``
     - ``(size_t)``
     - Local context ID for the specified |br|
       process member of a group
       
   * - ``PMIX_GROUP_INFO`` |br| ``pmix.grp.info``
     - ``pmix_data_array_t``
     - Array of pmix_info_t containing data |br|
       that is to be shared across all |br|
       members of a group during group |br|
       construction

   * - ``PMIX_IOF_TAG_DETAILED_OUTPUT`` |br| ``pmix.iof.tagdet``
     - ``(bool)``
     - Tag output with the |br|
       [local jobid,rank][hostname:pid] |br|
       and channel it comes from
       
   * - ``PMIX_IOF_TAG_FULLNAME_OUTPUT`` |br| ``pmix.iof.tagfull``
     - ``(bool)``
     - Tag output with the [nspace,rank] |br|
       and channel it comes from
       
   * - ``PMIX_LOG_AGG`` |br| ``pmix.log.agg``
     - ``(bool)``
     - Whether to aggregate and prevent |br|
       duplicate logging messages based |br|
       on key value pairs.
         
   * - ``PMIX_LOG_KEY`` |br| ``pmix.log.key``
     - ``(char*)``
     - key to a logging message
         
   * - ``PMIX_LOG_VAL`` |br| ``pmix.log.val``
     - ``(char*)``
     - value to a logging message
         
   * - ``PMIX_MYSERVER_URI`` |br| ``pmix.mysrvr.uri``
     - ``(char*)``
     - URI of this proc's listener socket
         
   * - ``PMIX_QUALIFIED_VALUE`` |br| ``pmix.qual.val``
     - ``(pmix_data_array_t*)``
     - Value being provided consists of the |br|
       primary key-value pair in first position, |br|
       followed by one or more key-value |br|
       qualifiers to be used when |br|
       subsequently retrieving the primary |br|
       value
         
   * - ``PMIX_WDIR_USER_SPECIFIED`` |br| ``pmix.wdir.user``
     - ``(bool)``
     - User specified the working directory
         
   * - ``PMIX_RUNTIME_OPTIONS`` |br| ``pmix.runopt``
     - ``(char*)``
     - Environment-specific runtime |br|
       directives that control job behavior
         
   * - ``PMIX_ABORT_NON_ZERO_TERM`` |br| ``pmix.abnz``
     - ``(bool)``
     - Abort the spawned job if any process |br|
       terminates with non-zero status
         
   * - ``PMIX_DO_NOT_LAUNCH`` |br| ``pmix.dnl``
     - ``(bool)``
     - Execute all procedures to prepare the |br|
       requested job for launch, but do not |br|
       launch it. Typically combined with the |br|
       PMIX_DISPLAY_MAP or |br|
       PMIX_DISPLAY_MAP_DETAILED for |br|
       debugging purposes.
         
   * - ``PMIX_SHOW_LAUNCH_PROGRESS`` |br| ``pmix.showprog``
     - ``(bool)``
     - Provide periodic progress reports on |br|
       job launch procedure (e.g., after |br|
       every 100 processes have been |br|
       spawned)
         
   * - ``PMIX_AGGREGATE_HELP`` |br| ``pmix.agg.help``
     - ``(bool)``
     - Aggregate help messages, reporting |br|
       each unique help message once |br|
       accompanied by the number of |br|
       processes that reported it
         
   * - ``PMIX_REPORT_CHILD_SEP`` |br| ``pmix.rptchildsep``
     - ``(bool)``
     - Report the exit status of any child |br|
       jobs spawned by the primary job |br|
       separately. If false, then the final |br|
       exit status reported will be zero if the |br|
       primary job and all spawned jobs exit |br|
       normally, or the first non-zero status |br|
       returned by either primary or child |br|
       jobs.
         
   * - ``PMIX_DISPLAY_MAP_DETAILED`` |br| ``pmix.dispmapdet``
     - ``(bool)``
     - display a highly detailed placement |br|
       map upon spawn
       
   * - ``PMIX_DISPLAY_ALLOCATION`` |br| ``pmix.dispalloc``
     - ``(bool)``
     - display the resource allocation
         
   * - ``PMIX_DISPLAY_TOPOLOGY`` |br| ``pmix.disptopo``
     - ``(char*)``
     - comma-delimited list of hosts whose |br|
       topology is to be displayed
         
   * - ``PMIX_DISPLAY_PROCESSORS`` |br| ``pmix.dispcpus``
     - ``(char*)``
     - comma-delimited list of hosts whose |br|
       available CPUs are to be displayed

   * - ``PMIX_DISPLAY_PARSEABLE_OUTPUT`` |br| ``pmix.dispparse``
     - ``(bool)``
     - display requested info in a format |br|
       more amenable to machine parsing

   * - ``PMIX_SORTED_PROC_ARRAY`` |br| ``pmix.sorted.parr``
     - ``(bool)``
     - Proc array being passed has been |br|
       sorted
         
   * - ``PMIX_QUERY_PROVISIONAL_ABI_VERSION`` |br| ``pmix.qry.prabiver``
     - ``(char*)``
     - The PMIx Standard Provisional ABI |br|
       version(s) supported, returned in the |br|
       form of a comma separated list of |br|
       "MAJOR.MINOR" pairs

   * - ``PMIX_QUERY_STABLE_ABI_VERSION`` |br| ``pmix.qry.stabiver``
     - ``(char*)``
     - The PMIx Standard Stable ABI |br|
       version(s) supported, returned in the |br|
       form of a comma separated list of |br|
       "MAJOR.MINOR" pairs

.. note:: The attribute ``PMIX_DEBUG_STOP_IN_APP`` has been modified
          to only support a ``PMIX_BOOL`` value instead of an optional
          array of ranks due to questions over the use-case calling
          for stopping a subset of a job's processes while allowing
          others to run "free".

Datatypes
---------

* ``pmix_value_cmp_t``: an enum indicating the relative value of
  two ``pmix_value_t objects``. Values include:

  * ``PMIX_EQUAL``
  * ``PMIX_VALUE1_GREATER``
  * ``PMIX_VALUE2_GREATER``
  * ``PMIX_VALUE_TYPE_DIFFERENT``
  * ``PMIX_VALUE_INCOMPATIBLE_OBJECTS``
  * ``PMIX_VALUE_COMPARISON_NOT_AVAIL``

* ``pmix_boolean_t``: an enum indicating boolean state of a
  ``pmix_value_t`` (possibly contained in a ``pmix_info_t`` object):

  * ``PMIX_BOOL_TRUE``
  * ``PMIX_BOOL_FALSE``
  * ``PMIX_NON_BOOL``

* ``pmix_disk_stats_t``: contains statistics on disk read/write operations
* ``pmix_net_stats_t``: contains statistics on network activity
* ``pmix_node_stats_t``: contains statistics on node resource usage
* ``pmix_proc_stats_t``: contains statistics on process resource usage


Datatype static initializers
^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Static initializers were added for each complex data type (i.e., a data type
defined as a struct). Most are contained in the Standard, but the following
extensions have been provided:

* ``PMIX_PROC_STATS_STATIC_INIT``
* ``PMIX_DISK_STATS_STATIC_INIT``
* ``PMIX_NET_STATS_STATIC_INIT``
* ``PMIX_NODE_STATS_STATIC_INIT``


Macros
------
Although the convenience macros have been deprecated, several were
added (in deprecated form) that previously were missing. These
are added for symmetry to support those who continue to use
the macros, and include:

* ``PMIX_XFER_PROCID``: transfer a ``pmix_proc_t`` to another one
  (non-destructive copy)
* ``PMIX_INFO_SET_END``: mark this ``pmix_info_t`` as being at the end
  of an array
* ``PMIX_INFO_SET_PERSISTENT``: mark that the data in this
  ``pmix_info_t`` is not to be released by ``PMIX_Info_destruct()`` (or its
  macro form)
* ``PMIX_INFO_SET_QUALIFIER``: mark this ``pmix_info_t`` as a qualifier to the
  primary key
* ``PMIX_INFO_IS_PERSISTENT``: test if this ``pmix_info_t`` has been marked as persistent
* ``PMIX_INFO_IS_QUALIFIER``: test if this ``pmix_info_t`` has been marked as a qualifier
* ``PMIX_DATA_ARRAY_INIT``: initialize a ``pmix_data_array_t``
* ``PMIX_CHECK_TRUE``: check if a ``pmix_value_t`` is boolean ``true`` (supports
  string as well as traditional boolean values)
* ``PMIX_CHECK_BOOL``: check if a ``pmix_value_t`` is a boolean value (supports
  string as well as traditional boolean values)


Macros supporting ``pmix_disk_stats_t`` objects:

* ``PMIX_DISK_STATS_CONSTRUCT``
* ``PMIX_DISK_STATS_CREATE``
* ``PMIX_DISK_STATS_DESTRUCT``
* ``PMIX_DISK_STATS_FREE``
* ``PMIX_DISK_STATS_RELEASE``

Macros supporting ``pmix_net_stats_t`` objects:

* ``PMIX_NET_STATS_CONSTRUCT``
* ``PMIX_NET_STATS_CREATE``
* ``PMIX_NET_STATS_DESTRUCT``
* ``PMIX_NET_STATS_FREE``
* ``PMIX_NET_STATS_RELEASE``

Macros supporting ``pmix_node_stats_t`` objects:

* ``PMIX_NODE_STATS_CONSTRUCT``
* ``PMIX_NODE_STATS_CREATE``
* ``PMIX_NODE_STATS_DESTRUCT``
* ``PMIX_NODE_STATS_RELEASE``

Macros supporting ``pmix_proc_stats_t`` objects:

* ``PMIX_PROC_STATS_CONSTRUCT``
* ``PMIX_PROC_STATS_CREATE``
* ``PMIX_PROC_STATS_DESTRUCT``
* ``PMIX_PROC_STATS_FREE``
* ``PMIX_PROC_STATS_RELEASE``
