The poe command enables the user to load and execute programs on different nodes. The poe acts like mpirun or mpprun on other platforms.

When you start a program with poe, you want to load a number of instances of it (tasks) on the resources you requested. The resources may be on one node or on different nodes. To do this, follow the poe command with the program name and any options. These options can include program options, followed by any of command line options of poe. If the program was compiled and linked with the mpXXX compilers, the poe command can be omitted. Thus, the following two commands are equivalent for these programs:

$ poe a.out [options to a.out ...] [poe options ...]
$ a.out [options to a.out ... ] [poe options ...] 

The poe command takes command line options that define resources like number of MPI tasks or network. These options are given on the command line when running an MPI program interactively. Alternatively, they can be set via environment variables. POE environment variables share the common prefix MP_. These options take defaults if they are not given. When running an MPI program in batch using LoadLeveler, many of the poe options become overridden by corresponding LoadLeveler keywords.

Table 7.1 shows the most important poe options to get started.

The following examples comment on the use of poe interactively or from within a batch job. Chapter 10, Examples gives additional working examples including short source codes for sample programs.

Example 7.1 shows how to invoke instances of the uname command on different nodes. Three variants are given, which do the same. The first calls poe interactively with command line flags. The second calls poe interactively with environment variables. The third calls poe from within a LoadLeveler script.

$ poe uname -n -s -rmpool 0 -nodes 1 -tasks_per_node 2 -labelio yes -stdoutmode ordered
Interactive, poe environment variables:
$ export MP_RMPOOL=0
$ export MP_NODES=1
$ export MP_TASKS_PER_NODE=2
$ export MP_LABELIO=yes
$ export MP_STDOUTMODE=ordered
$ poe uname -n -s
      

#!/bin/ksh
# @ job_type       = parallel
# @ node           = 1
# @ tasks_per_node = 2
# @ resources      = ConsumableCpus(1) ConsumableMemory(16 mb)
# @ output         = poe_ex1.llout
# @ error          = $(output)
# @ class          = cdev
# @ queue

poe uname -n -s -labelio yes -stdoutmode ordered

   
0:AIX hreg02a-en0
1:AIX hreg02a-en0

Example 7.2 shows how to invoke an MPI program with a number of total tasks. The control on how the tasks are allocated is given to LoadLeveler. Use of the HPS adapters will be requested. The shown environment settings are critical for getting performance. The shown input is appropriate for the majority of MPI applications. It is assumed that the executable is named a.out, and that it needs an input file called input.in as argument.

$ export MP_SHARED_MEMORY=yes
$ export MP_WAIT_MODE=poll
$ export MP_SINGLE_THREAD=yes
$ a.out input.in -rmpool 0 -procs 6 -euidevice sn_all -euilib us

#!/bin/ksh
# @ job_type         = parallel
# @ total_tasks      = 6
# @ blocking         = unlimited
# @ network.mpi      = sn_all,,us
# @ resources        = ConsumableCpus(1) ConsumableMemory(1968 mb)
# @ output           = poe_ex2.llout
# @ error            = $(output)
# @ environment      = MEMORY_AFFINITY=MCM; \
#                      MP_SHARED_MEMORY=yes; MP_WAIT_MODE=poll; \
#                      MP_SINGLE_THREAD=yes; MP_TASK_AFFINITY=MCM
# @ wall_clock_limit = 70,60
# @ node_usage       = shared
# @ queue

./a.out input.in

Most SMP applications follow the OpenMP standard. Their runtime behaviour depends on the setting of some environment variables. There exist the standardized OMP_XXXX variables. On AIX, there also exists a set of XLSMPOPTS settings that do the same. If both are set, the OpenMP variable takes precedence.