PGHPF 3.0 is the second release of PGI's native (HPF-to-assembly code) High Performance Fortran compiler for IA-32 processor-based systems. All features of Full HPF 1.1 and Fortran 90 are supported, with the few exceptions noted in this document. In addition, several HPF 2.0 features have been added in this release. See section 3.1 below for a list of new features. If you encounter any feature that is not supported, and not listed in section 3.3, Restrictions, please consider it a bug and report it to PGI at the e-mail address trs@pgroup.com.
The following features have been added to PGHPF 3.0 for IA-32 systems:
* Support for GEN_BLOCK data distributions has been added.
* Support for distributed components of derived types has been added.
The following important features, first available in interim release 1.7-6, are worth re-emphasizing here:
* Support for HPF 2.0 semantics is under control of the -Mhpf2 command-line switch. This switch will become the default in a future release of PGHPF.
* The HPF 2.0 SHADOW directive is supported.
* As specified in HPF 2.0, Fortran 90 POINTER mappings must conform to the TARGET mapping.
* The HPF 2.0 REDUCTION clause on INDEPENDENT loops is supported.
* The compiler switch -Mbyteswapio is now supported by PGHPF. It will cause bytes to be swapped from big-endian to little-endian format (or vice versa) on input/output of unformatted Fortran binary files.
* A new runtime switch -pghpf -system_clock_rate allows the user to set the granularity of the clock used in the F90 SYSTEM_CLOCK intrinsic. By default, the granularity is 1000 ticks per second. Most platforms support higher resolution, but implement the timer in only 32 bits. For example, a value of -pghpf -system_clock_rate 1000000 could be used to increase granularity to 1,000,000 ticks per second, but the clock would roll over after approximately 30 minutes. A corresponding environment variable PGHPF_SYSTEM_CLOCK_RATE can be set to increase the granularity for all runs. This option is available to allow users to perform high resolution timings without sacrificing the occasional need to time computations that take hours or even days.
Most of these features are described in more detail either in the following sections or in the PGHPF Compiler User's Guide.
This section provides additional information on some of the new features listed above.
The GEN_BLOCK distribution format is an approved HPF 2.0 extension that facilitates generalized block distributions. Arbitrarily-sized contiguous segments of an array can be mapped directly onto the available processors. The sizes of these segments are specified by values in a user-defined integer mapping array, one value per processor. For example:
subroutine test (a, ia)
integer ia(number_of_processors())
real a(1000)
!hpf$ distribute (gen_block(ia)) :: a
. . .
The GEN_BLOCK facility provides functionality for simple and efficient implementations of Adaptive Grid Algorithms.
NOTE 1: GEN_BLOCK-distributed arrays cannot be the subject of a REDISTRIBUTE directive, and hence should not be declared DYNAMIC. However, it is legal to pass a GEN_BLOCK-distributed array as an actual argument to a subroutine where the corresponding dummy argument is also GEN_BLOCK-distributed but has different values in the mapping array. That is, GEN_BLOCK arrays can be re-distributed at subroutine boundaries. Also, GEN_BLOCK-distributed arrays can be re-distributed by assignment to another GEN_BLOCK-distributed array which has different values in the mapping array.
NOTE 2: As with CYCLIC-distributed arrays, GEN_BLOCK-distributed arrays are allocated without SHADOW regions. This is as opposed to BLOCK-distributed arrays, which by default are allocated with SHADOW regions of size 4. A default shadow size may be added for GEN_BLOCK-distributed arrays in a future release of PGHPF.
The PGHPF 3.0 compiler supports the HPF 2.0 approved extension for distribution of components of derived types. For example, the following is legal:
. . .
type velocity
real x(n)
real y(n)
real z(n)
!hpf$ distribute (block) :: x
!hpf$ align (:) with x(:) :: y, z
end type velocity
. . .
Either a DISTRIBUTE or an ALIGN directive can appear within the derived type definition. A TEMPLATE directive can not appear within the derived type definition, but components of a derived type can be aligned to a globally declared TEMPLATE.
NOTE: Components of derived types can not be declared DYNAMIC. DYNAMIC is included as part of the HPF 2.0 approved extension for distributed components of derived types, but this feature is not supported in PGHPF 3.0.
The -Mhpf2 command-line option causes the PGHPF compiler to generate code assuming HPF 2.0 semantics. In particular, HPF 2.0 requires that if an array is to be re-distributed at a subroutine boundary (i.e. the distribution of the actual argument differs from that of the corresponding dummy argument), then the calling subroutine must contain an INTERFACE block which specifies the new distribution. If no INTERFACE block exists, the results of such a call are undefined.
This new requirement on the part of the HPF programmer allows for caller re-mapping, whereby all data re-mapping is performed in the calling routine rather than in the callee. In the event that no re-mapping takes place, the HPF compiler can eliminate costly checks that were required in order to support HPF 1.1 semantics.
It is recommended that you use -Mhpf2 whenever possible. In general, it should improve performance. -Mhpf2 will become the default mode of compilation in a future release of PGHPF.
For block-distributed dimensions of arrays declared in global HPF procedures, shadow areas are allocated on the lower and upper ends of each processor's local block. The shadow size is determined by the SHADOW directive if one is present for the array. Otherwise a default shadow size is computed by the compiler just as in previous PGHPF releases. The lower and upper shadow sizes may be specified (or computed) independently and need not be the same.
The HPF 2.0 SHADOW directive is fully recognized in PGHPF 3.0. A typical usage might appear like this:
!hpf$ distribute (*,block) :: a, b, c
!hpf$ shadow (0,1:2) :: a, b, c
To specify that there should be no shadow regions in the first dimension, which is collapsed, but that a 1 element shadow region should be allocated at the lower end of the local block in the second dimension and a 2 element shadow region should be allocated at the upper end of the local block in the second dimension. If the shadow region is of uniform size at either end of the local block, say 2 on both ends, the following syntax can be used:
!hpf$ distribute (*,block) :: a, b, c
!hpf$ shadow (0,2) :: a, b, c
The default shadow size limit is set using the -Moverlap=size:<n> command-line option. If this option is not used, then the default shadow size limit is 4. This is the upper limit on the shadow size computed by the compiler in the absence of a shadow directive. Differencing codes which require a shadow region larger than 4 must either have shadow directives inserted or use the -Moverlap option in order for the PGHPF compiler to generate code using the overlap shift optimization (see Chapter 6 of the PGHPF Compiler User's Guide for more information on this optimization).
If an array is passed as an actual argument in a procedure call, then the default lower and upper shadow size for each of its block distributed dimensions is set to the default shadow size limit, regardless of any shadow directive for the dummy argument. Otherwise, the default shadow sizes are determined by analysis of the subscript expressions in array references in FORALL statements, array assignments, and DO INDEPENDENT loops. The computed default shadow size will be less than or equal to the default shadow size limit.
The shadow directive for a dummy array argument must appear in the specification part of a procedure if either the procedure is contained in a module or if it is a global HPF procedure and the -Mhpf2 command-line option is not used. Otherwise, the shadow directive for a dummy array argument must appear in an INTERFACE block visible to the caller and any shadow directive for the dummy argument appearing in the procedure body is ignored.
On entry to an HPF procedure, including an HPF_LOCAL extrinsic, a dummy array will have shadow areas no smaller than the size specified in the shadow directive, if present. If the actual argument has larger shadow areas and there is no other reason for the argument to be copied, then the larger shadow areas will be preserved. This minimizes argument copying at the procedure interface. The values in the shadow area of a dummy array are undefined on entry to and exit from a procedure.
In an HPF_LOCAL extrinsic procedure, the values returned by the lbound, ubound, shape, and size intrinsics exclude the shadow areas of a dummy argument. Thus these intrinsics do not indicate the actual allocated bounds or shape of a dummy array. It is safe to under-index and over-index a dummy array to the extent specified in the interface shadow directive, as long as the HPF_LOCAL extrinsic procedure was called from a global HPF program unit.
To explicitly make all shadow regions coherent, for example just prior to entering an HPF_LOCAL extrinsic in which the shadow regions will be manipulated directly using over- and under-indexing, you can use a simple code fragment such as the following. For example, a, b, and c in the second example above could be made coherent using the following fragment:
work = cshift (a,2,dim=2) + cshift (a,-2,dim=2) +
& cshift (b,2,dim=2) + cshift (b,-2,dim=2) +
& cshift (c,2,dim=2) + cshift (c,-2,dim=2)
After the above fragment is executed, all of the shadow regions will be up-to-date with the current values of the corresponding array locations on neighboring processors.
NOTE: In PGHPF 3.0, the above technique will only work correctly if the size of the distributed dimension(s) is an even multiple of the number of processors in the corresponding dimension(s) of the processor grid. This restriction will be lifted in a future release of PGHPF.
In PGHPF 3.0, the syntax of the INDEPENDENT directive has been extended to support the REDUCTION clause as specified in the HPF 2.0 language definition. The syntax is now as follows:
!HPF$ INDEPENDENT [,NEW(new-vars)]
!HPF$&
[,REDUCTION(reduction-vars)]
See the PGHPF Compiler User's Guide or the High Performance Fortran Handbook for a complete description of the INDEPENDENT directive and the NEW clause.
The REDUCTION clause asserts that the named variables are updated in the INDEPENDENT loop by a series of operations that are associative and commutative. Furthermore, the intermediate values of the REDUCTION variables are not used within the loop except in the updates themselves. Thus, the value of the REDUCTION variable after the loop may be computed as the result of a reduction tree. Use of this clause allows for parallelization of reduction loops which would not otherwise be considered INDEPENDENT.
For example, the following loop can be parallelized using INDEPENDENT with the REDUCTION clause:
!HPF$ INDEPENDENT, REDUCTION(SUM)
DO I = 1, N
SUM = SUM + A(I)
ENDDO
Whereas specifying the loop to be INDEPENDENT without the REDUCTION clause would be incorrect. See the High Performance Fortran Language Specification Version 2.0 for a more detailed discussion of the REDUCTION clause.
This section lists Fortran 90 and HPF features that are not supported in PGHPF 3.0.
integer, pointer :: p
integer, target :: a(10),b(10)
!hpf$ distribute (block) :: a
p => a(1) ! unsupported
p => b(1) ! supported
end
CSHIFT() |
RESHAPE() |
TRANSFER() |
EOSHIFT() |
SHAPE() |
TRANSPOSE() |
LBOUND() |
SIZE() |
UBOUND() |
MERGE() |
SPREAD() |
UNPACK() |
PACK() |
COPY_PREFIX() |
COPY_SCATTER() |
COPY_SUFFIX() |
HPF_ALIGNMENT() |
HPF_DISTRIBUTE() |
HPF_TEMPLATE() |
The HPF_LIBRARY routines GRADE_UP and GRADE_DOWN require a DIM argument. These routines also do not support CYCLIC or GEN_BLOCK distributions of the selected dimension.
In evaluating a character string expression on the right-hand side of an assignment, the values on the left-hand side may not be used. This is allowed in Fortran 90, but not currently supported in PGHPF 3.0. For example, the following will not be valid:
DATE (2:5) = DATE(1:4)
The PGHPF 3.0 implementation of PURE conforms to the HPF 1.1 language specification, with the following exception: in PURE subroutines PGHPF will not generate any communication for distributed COMMON variables or distributed MODULE variables. The user is advised to pass distributed COMMON as arguments to a PURE subroutine, or use non-distributed COMMON .
An optional argument should not be used as an align-target for any variable that is not also optional. If an alignee is present, then the align-target must also be present.
For example:
subroutine test1(a,b)
integer, dimension(10):: a,b
optional :: a
!hpf$ template t(n)
!hpf$ distribute (block)::t
!hpf$ align a(i) with t(i)
!hpf$ align b(i) with a(i) ! THIS IS A PROBLEM
Should be rewritten as:
subroutine test1(a,b)
integer, dimension(10):: a,b
optional :: a
!hpf$ template ta(n)
!hpf$ distribute (block):: ta
!hpf$ align a(i) with ta(i)
!hpf$ template tb(n)
!hpf$ distribute (block):: tb
!hpf$ align b(i) with tb(i) ! THIS IS FINE
The following compile-time warning message:
PGHPF-W-301-Non-replicated mapping for character/struct/union array, char_table, ignored (file.F: lineno)
Indicates that PGHPF 3.0 ignores distribution directives applied to character types, arrays subject to a SEQUENCE directive, and NAMELIST arrays.
PGI currently supports (i.e. can use as the underlying communication mechanism) version 1.10 of MPI For information on obtaining the current public domain version of MPI, contact the following:
http://www.mcs.anl.gov/mpi/index.html