MPI_Waitall(3) Manual Page

If one or more of the requests completes with an error, MPI_ERR_IN_STATUS is returned. An error value will be present is elements of array_of_statuses associated with the requests. Likewise, the MPI_ERROR field in the status elements associated with requests that have successfully completed will be MPI_SUCCESS. Finally, those requests that have not completed will have a value of MPI_ERR_PENDING.

While it is possible to list a request handle more than once in the array_of_requests, such an action is considered erroneous and may cause the program to unexecpectedly terminate or produce incorrect results.

The 'MPI_ERROR' field of the status return is only set if the return from the MPI routine is 'MPI_ERR_IN_STATUS'. That error class is only returned by the routines that take an array of status arguments ('MPI_Testall', 'MPI_Testsome', 'MPI_Waitall', and 'MPI_Waitsome'). In all other cases, the value of the 'MPI_ERROR' field in the status is unchanged. See section 3.2.5 in the MPI-1.1 specification for the exact text.

For send operations, the only use of status is for 'MPI_Test_cancelled' or in the case that there is an error in one of the four routines that may return the error class 'MPI_ERR_IN_STATUS', in which case the 'MPI_ERROR' field of status will be set. In that case, the value will be set to 'MPI_SUCCESS' for any send or receive operation that completed successfully, or 'MPI_ERR_PENDING' for any operation which has neither failed nor completed.

This routine is thread-safe. This means that this routine may be safely used by multiple threads without the need for any user-provided thread locks. However, the routine is not interrupt safe. Typically, this is due to the use of memory allocation routines such as 'malloc' or other non-MPICH runtime routines that are themselves not interrupt-safe.

All MPI routines in Fortran (except for 'MPI_WTIME' and 'MPI_WTICK') have an additional argument 'ierr' at the end of the argument list. 'ierr' is an integer and has the same meaning as the return value of the routine in C. In Fortran, MPI routines are subroutines, and are invoked with the 'call' statement.

All MPI objects (e.g., 'MPI_Datatype', 'MPI_Comm') are of type 'INTEGER' in Fortran.

All MPI routines (except 'MPI_Wtime' and 'MPI_Wtick') return an error value; C routines as the value of the function and Fortran routines in the last argument. Before the value is returned, the current MPI error handler is called. By default, this error handler aborts the MPI job. The error handler may be changed with 'MPI_Comm_set_errhandler' (for communicators), 'MPI_File_set_errhandler' (for files), and 'MPI_Win_set_errhandler' (for RMA windows). The MPI-1 routine 'MPI_Errhandler_set' may be used but its use is deprecated. The predefined error handler 'MPI_ERRORS_RETURN' may be used to cause error values to be returned. Note that MPI does not guarantee that an MPI program can continue past an error; however, MPI implementations will attempt to continue whenever possible.

The MPI Standard does not specify what the result of the multiple completion routines is when an error occurs. For example, in an 'MPI_WAITALL', does the routine wait for all requests to either fail or complete, or does it return immediately (with the MPI definition of immediately, which means independent of actions of other MPI processes)? MPICH has chosen to make the return immediate (alternately, local in MPI terms), and to use the error class 'MPI_ERR_PENDING' (introduced in MPI 1.1) to indicate which requests have not completed. In most cases, only one request with an error will be detected in each call to an MPI routine that tests multiple requests. The requests that have not been processed (because an error occurred in one of the requests) will have their 'MPI_ERROR' field marked with 'MPI_ERR_PENDING'. * MPI_ERR_REQUEST - Invalid 'MPI_Request'. Either null or, in the case of a 'MPI_Start' or 'MPI_Startall', not a persistent request. * MPI_ERR_OTHER - Other error; use 'MPI_Error_string' to get more information about this error code.