Comments on: Job Scheduling with Torque

By: University Research Computing » Update on Transition to Torque/Maui

University Research Computing » Update on Transition to Torque/Maui — Tue, 01 Sep 2009 13:18:53 +0000

[...] so, please begin submitting your cluster jobs using the new system as described on our web pages at http://www.urc.uncc.edu/urc/faqs/job-scheduling-with-torque/. We have provided example scripts for our most popular applications (see folder [...]

By: University Research Computing » How do I Submit a Job?

University Research Computing » How do I Submit a Job? — Tue, 18 Aug 2009 16:38:05 +0000

[...] for job scheduling, so we would encourage you to use start using Torque instead of Condor. Here is more information about submitting jobs to Torque on the URC cluster.Jobs are submitted to Condor using a command line interface that requires a [...]

By: University Research Computing » Transition to Torque/Maui

University Research Computing » Transition to Torque/Maui — Mon, 22 Jun 2009 14:04:46 +0000

[...] simple, but does require learning a few new techniques. The URC staff has prepared a web page with instructions for getting started with torque, as well as, sample torque scripts for many of our most common applications (see folder [...]

By: Jonathan Halter

Jonathan Halter — Mon, 18 May 2009 21:20:25 +0000

Hi Jeff,

Thanks for the comments. I work with Mike and thought I’d reply back to your comment.

We certainly know about Open MPI’s built-in Torque support. In fact we did compiled Torque support into Open MPI for our testing. It worked great and we had no problems with it; this is probably the direction we will go for the next production version we make available on the cluster.

But the current version of Open MPI we have available on the cluster was built before we had Torque available on our systems, so the support was not compiled in. We figured that during the transition from Condor to Torque/Maui, we’d leave the various MPI implementations alone, so that the syntax would remain consistent between them, then we would upgrade Open MPI with the added functionality after the dust settled.

By: Jeff Squyres

Jeff Squyres — Mon, 18 May 2009 20:49:18 +0000

Mike –

My name is Jeff Squyres; I’m one of the core Open MPI developers. Google Alerts found this web page for me.

Just curious; you cite above that Open MPI uses the “-hostfile” approach when running jobs in Torque. Did you guys not build the Torque support in Open MPI for some reason? If there’s something wrong with Open MPI’s Torque support, please let us know.

FWIW, Open MPI supports the Torque system such that mpirun will directly query Torque to find out what hosts to use and how many MPI processes to launch on each. Further, Open MPI will use Torque’s native launcher (the “TM” system) rather than rsh/ssh to launch MPI processes on remote nodes. This can provide better job cleanup in failure cases, and provide better accounting details, too.

For example:

#! /bin/bash
# ===== PBS OPTIONS =====
### Set the job name
#PBS -N MyJob

### Run in the queue named “urc”
#PBS -q urc
### Specify the number of cpus for your job.
#PBS -l nodes=16:ppn=4

mpirun my_mpi_prgram

will launch my_mpi_program on 4 cores of each of 16 nodes (a benefit of using the Torque Open MPI support is that you don’t need to specify the -np value on the mpirun command line — it’ll automatically figure it out based on the information it receives from Torque).