Hybrid MPI + OpenMP

At hexagon you can run a hybrid MPI + OpenMP job, by running MPI between the nodes and OpenMP within the nodes.


No special compiler directives are needed to activate MPI, but to activate openMP directives, compile and link with


Fortran:
-mp=nonuma     for the PGI compiler, and
-mp                for the Pathscale compiler

C and C++:
-mp                for the PGI compiler, and
-mp                for the Pathscale compiler

In the batch-script set

export OMP_NUM_THREADS=threads_per_node

mppdepth=threads_per_node

mppwidth=number_of_nodes

These numbers should correspond to

aprun ... -n number_of_nodes -d threads_per_node ...