|
The primary compiler on the Cluster system is the IBM Pathscale compiler which provides methods for 'C', 'C++' and various specifications of Fortran.
IBM e1350 Cluster
- 'C' Compiler: pathcc version 2.9.99
- 'C++' Compiler: pathCC version 2.9.99
- 'C' and 'C++' Compiler: gcc (GCC) 3.3.3 (SuSE Linux)
- Fortran90/77 Compiler: pathf90 version 2.9.99
IBM P690 SMP (x2) (logon to host "chpcc162" to access directly)
- 'C' and 'C++' Compiler: gcc (GCC) 3.3.3 (SuSE Linux)
- Fortran90/77 Compiler: IBM XL Fortran Advanced Edition V10.1 for Linux
IBM Blue Gene/P
- Based on GCC 4.1.2: gcc, g++, gfortran
- IBM XL C/C++ Advanced Edition for Blue Gene/P, V9.0.
- IBM Fortran: mpixlf77, mpixlf2003, mpixlf90, mpixlf95
Common to the Cluster environment is the Message Passing Interface (MPI) and the preferred implementation is that of MVAPICH [1] of which the most recent release is 0.9.9. The MVAPICH is an implementation specifically built to make use of the Infiniband connectivity and it is essential that your codes link with these libraries to get the best performance. Alternatives for MVAPICH are MPICH which is built with the Pathscale compiler but will not make use of the Infiniband interconnect but will rather use the 1Gb ethernet TCP/IP stack.
| Interaction matrix |
| Interconnect |
InfiniBand 10Gb |
TCP/IP 1Gb |
| Compiler |
Pathscale |
GNU GCC |
| MPI |
MVAPICH |
MPICH or MVAPICH |
In addition to MPI libraries there are also some maths libraries available namely GSL, FFTW, GotoBLAS/LAPACK and ACML.
For a full list of available libraries and compilers see “Compilers Libraries.txt” in chpcln:/CHPC/usr/local/doc/QUICKSTART/
Compilers and Libraries
| Code name |
version |
directory |
Notes |
| gcc |
v3.3.3 |
|
32-bit now enabled as well as 64-bit |
| MVAPICH |
0.9.8 |
/CHPC/usr/local/mvapich |
with Pathscale compiler |
| MVAPICH |
0.9.8 |
/CHPC/usr/local/mvapich/gcc |
with GNU gcc compiler |
| MPICH |
1.2.7p1 |
/CHPC/usr/local/mpich/bin |
Standard mpich built with pathscale |
| MVAPICH |
0.9.8 |
/CHPC/usr/local/mvapich_098_nsu |
Standard mvapich with Pathscale and -fno-second-underscore |
| MVAPICK |
0.9.9.beta |
/CHPC/usr/local/mvapich_099b |
Beta version of mvapaich for use with NAMD |
| MPICH |
1.2.7p1 |
/CHPC/usr/local/mpich-32 |
32-bit version of MPI with pathscale |
| FFTW |
2.1.5 |
/CHPC/usr/local/fftw2.1.5 |
Single and double precision
mpi version built with mvapich v0.9.8 pathscale |
| FFTW |
3.1.2 |
/CHPC/usr/local/fftw3.1.2 |
Single and double precision |
| ACML |
3.6.0 |
/CHPC/usr/local/acml3.6.0 |
pathscale64 & gnu64 directories are 64bit precision
pathscale32 & gnu32 directories are 32bit precision |
| LAPACK |
3.1.1 |
/CHPC/usr/local/lapack |
liblapack-path-64.a compiled using pathscale 64bit
liblapack-path-32.a compiled using pathscale 32bit |
| HDF5 |
1.6.5 |
/CHPC/usr/local/hdf5-1.6.5 |
built with gcc, 64bit |
| GSL |
1.9 |
/CHPC/usr/local/gsl-1.9 |
built with gcc, 64-bit |
In addition in my home directory /CHPC/home/nallsopp you will find the Examples.tar.gz and a few example loadleveller scripts. If you want an example of a loadleveller script
for going over infiniband with MVAPICH take a look at hello_mpi.ll. If you want to go over IP with MPICH then take a look at run_hello_ib.ll, but please remember to build and r
un with the appropriate mpif90, mpicc, mpirun etc.
Also, you will notice that there are various new classed which have been created. You can see what is available by typing: llclass |
The IBM XL Compiler
From the man page:
More information is available at IBM's support site or IBM Infocenter
| bgxlc(1) IBM XL C/C++ Advanced Edition for Blue Gene/P, V9.0 bgxlc(1) |
| NAME |
bgxlc, bgxlc_r, bgxlc++, bgxlc++_r, bgxlC, bgxlC_r, bgcc, bgcc_r,
bgc89, bgc89_r, bgc99 and bgc99_r commands - invoke the IBM XL C/C++
compiler for Blue Gene/P. |
| SYNTAX |
<invocation-command> [ <option> | <inputfile> ] ... |
| DESCRIPTION |
The invocation commands compile C and C++ source files.
The code is compiled on a standard Linux SLES platform to be run on a
Blue Gene system.
The commands and their recommended uses are described below.
| Invocations |
Usage (supported standards) |
| bgxlc, bgxlc_r |
Compile C source files.
(ANSI C89, ISO C99,
IBM language extensions) |
bgxlc++, bgxlc+++_r,
bgxlC, bgxlC_r |
Compile C++ source files. |
| bgcc, bgcc_r |
Compile legacy code that
does not conform to Standard
C. (pre-ANSI C) |
| bgc89, bgc89_r |
Compile C source files with
strict conformance to the
C89 standard. (ANSI C89) |
| blgc99, bgc99_r |
Compile C source files with
strict conformance to the
C99 standard. (ISO 99) |
The main difference between these commands is that they use different
default options (which are set in the configuration file
/etc/opt/ibmcmp/vac/bg/9.0/vac.cfg).
All invocations with a suffix of _r allow for thread-safe compilation.
Use these commands to create threaded applications or to link programs
that use multi-threading.
These commands also process assembler source files and object files.
The compiler calls the link editor to produce a single executable or
shared library, unless you specify the compiler options that prevent
object files from being linked. |
| The input file may have any of the following suffixes: |
.C, .cpp, .cxx,
.cc, .cp, .c++ |
C++ source file |
| .c |
C source file |
| .i |
Preprocessed C or C++ source file |
| .a |
Archive file |
| .o |
Object file for ld command |
| .s |
Assembler source file |
| .S |
Assembler source file that
needs preprocessing |
| .so |
Shared object file |
|
| OPTIONS |
Compiler options are categorized by their functions and may be used to
set the compiler behavior. Options can be flag options or keyword
options.
Keyword options are specified in one of the following ways:
-q<option>
-q<option>=<suboption>
-q<option>=<suboption>:<suboption>
Keyword options with no suboptions represent switches that may be
either on or off. -qx turns the switch on, and -qnox turns the switch
off. For example, -qsource tells the compiler to produce a source
listing and -qnosource tells the compiler not to produce a source
listing. |
Name Mangling
Name mangling is a mechanism by which names of functions, procedures, and common blocks from Fortran source files are converted into an internal representation when compiled into object files. For example, a Fortran subroutine called foo gets turned into the name "foo_" when placed in the object file. We do this to avoid name collisions with similar functions in other libraries. This makes mixing code from C, C++, and Fortran easier. Name mangling ensures that function, subroutine, and common-block names from a Fortran program or library do not clash with names in libraries from other programming languages. For example, the Fortran library contains a function named "access", which performs the same function as the function access in the standard C library. However, the Fortran library access function takes four arguments, making it incompatible with the standard C library access function, which takes only two arguments. If your program links with the standard C library, this would cause a symbol name clash. Mangling the Fortran symbols prevents this from happening. By default, we follow the same name mangling conventions as the GNU g77 compiler and libf2c library when generating mangled names. Names without an underscore have a single underscore appended to them, and names containing an underscore have two underscores appended to them. The following examples should help make this clear:
molecule -> molecule_
run_check -> run_check_ _
energy_ -> energy_ _ _ |
This behavior can be modified by using the -fno-second-underscore and the -fno-underscoring options to the pathf95 compiler. PGI Fortran and Intel Fortran's default policies correspond to our -fno-second-underscore option. Common block names are also mangled. Our name for the blank common block is the same as g77 (_BLNK_ _). PGI's compiler uses the same name for the blank common block, while Intel's compiler uses _BLANK_ _.
In general BEST PRACTICE for the cluster system is to use the following:
% export FC=pathf90
% export CC=pathcc
% export FFLAGS=”-fno-underscoring” |
If you run into a problem with double underscoring then use the mvapich*_nsu (no second underscore) library.
Scalable Codes On IBM Blue Gene/P and Sun Microsystems
Please click: here
to download scalable codes on IBM Blue Gene/P
Please click: here
to download scalable codes on Sun Microsystems Cluster
|
