running jobs

Moab WorkLoad Manager

All jobs on the GPU,SUN and Bluegene are scheduled by Moab Cluster Suite

How to submit on GPU:

1. Compile your code

2. Run your submit script. (Please click here to view example scripts for customization)

Please note that you need to use an MPI inorder to call an mpi program. The system installed MPI is under /GPU/opt/open-mpi-new/.

export the mpi using the following command or add the lines on your .profile

export PATH=/GPU/opt/open-mpi-new/bin:$PATH

export LD_LIBRARY_PATH=/GPU/opt/open-mpi-new/lib:$LD_LIBRARY_PATH

Partitions available on GPU:

1. C2070
2. C1060

Moab Job Submit:

msub  scriptname 

allows users to submit jobs directly to Moab.

Moab job submit on BlueGene/P:

1. Create a submit script (Please click here to view example scripts for customization)

On the submit script, the following variables have to be same.

1. ppn = number of processors to execute/simulate the job (Moab variable).
2. -np = number of processors to execute/simulate the job (MPI variable).

Please note that the user can only run on the following partitions/processors:

E.g. 32 or 64 or 128 or 256 or 512 processors.

job submit:

msub scriptname 

Allows users to submit directly to Moab.

Moab job submit on SUN:

1. Compile your code (Please click here on how to compile on SUN)

2. Run your submit script (Please click here to view example scripts for customization)

Partitions available on SUN

1. haprtertown
2. nehalem
3. westmere
4. dell
5. sparc
6. test
7. viz

Moab Job Submit:

msub  scriptname -l partition=partition-name 

Allows users to submit to Moab.

top of the page

How to cancel jobs

Cancel jobs on GPU,SUN and Bluegene:

mjobctl -c jobid

used to selectively cancel the specified job(s) (active, idle, or non-queued) from the queue

Debugging

mpirun_dbg.dbx, mpirun_dbg.ddd, mpirun_dbg.gdb

Monitoring

For monitoring on nodes use one of

• nmon
• vmstat
• top
• xloadl(X11)

and of course ps and free

top of the page