5 Directives specific to the MPI version

5.1 Invoking the ScaLAPACK driver

To ensure that the distributed data ScaLAPACK driver is invoked, a block of code beginning with the keyword ''NEWSCF'' and ending with the keyword ''END'' must be included as shown below, otherwise the code will default to using the older replicated data MPI driver.

          newscf
          <control directives>
          end

The format of the control directives is explained in Part 4 of the manual in the section titled: SCF Convergence - Alternate Driver.

5.2 The two parallel diagonalisers

The ScaLAPACK code has two different parallel diagonalisers that can be used. There is a default one that uses the standard BLACS/ScaLAPACK diagonalisation routines, and one that uses a ''Divide and Conquer'' approach.

The ''Divide and Conquer'' diagonliaser is generally much faster than the default, but very occasinally exhibits erratic behaviour, generating incorrect Eigenvalues. This manifests itself in the energy between SCF cycles fluctuating violently. For this reason, this diagonaliser is not currently the default.

To activate the ''Divide and Conquer'' diagonaliser, insert the line:

          parallel diag divide

in the input file. The keywords are all in A format.

NB: the ''Divide and Conquer'' diagonaliser requires roughly four times the memory of the standard diagonaliser.

5.3 The Taskfarming binary

The taskfarming binary is named GAMESS-UK-7.0/bin/gamess-uk.taskfarm.

Upon startup, the binary will expect to find a file named task.list in the directory that it is running in that contains a list of all the jobs to be run, together with any control directives.

The control directives are as follows:

5.3.1 GROUPSIZE

The first line of the task.list should be the keyword ''GROUPSIZE'' in A format, followed by an integer (in I format) specifying the number of processors making up a group. Each individual job will then be run on this number of processors. An exception to this occurs when the total number of processors minus one (for the server process) is not exactly divisible by the groupsize. In this case, one group will be created that will have fewer processors then the default group size.

5.3.2 IDLEG

The ''IDLEG'' directive is an optional directive that may be the second line in the task.list file and determines the percentage of groups that are permitted to be running idle before the taskfarm is halted. The directive consits of the keyword 'IDLEG'' in A format followed by an integer in I format, where the integer specifies the percentage of groups that may be idle.

Once all of the jobs in the task.list have been allocated to the different groups, as the processor groups will be runing idle until the last job has finished running. Without the use of IDLEG, it is potentially possible that several thousand processors could be sat burning up job time waiting for a 4-processor job to finish. Use of the ''IDLEG'' directive allows the user to decide when this cost becomes prohibitive and the job should be aborted.

NB: It should be noted that as it is currently implemented, using the IDLEG directive causes all running jobs to abort regardless of their state, so any restart information will be lost.

5.3.3 Format of the task.list file

The remaining lines in the task.list file should just be a list of the input files in the submission directory, without the .in suffix that is expected to be appended to all GAMESS-UK input files.

The input reader supports all of the functionality of the standard GAMESS-UK reader, so it is possible to comment out files from the list with a ''#'' symbol for example, or place multiple filenames on the same line, separating them with a backslash.

The input reader will stop processing inputs until it either encounters a line with more than five aterisks (''*****'') or encounters the end of the file.

5.3.4 Taskfarming output

For each input file with a .in suffix, a standard GAMESS-UK output file will be created with the same name but a .out suffix. Any other files (ed2, ed3, punchfile,... etc.) will be named after the input file together with an appropriate suffix.

In addition a file called taskfarm.summary will also be created. This lists the jobs that were run, the length of time they took and the outcome of the job, together with a summary of any files that could not be processed.

5.3.5 Sample input

The below is an example task.list file demonstarting all of the possible control directives.:

	  # This is a comment
	  #
	  # Run the jobs on 8 processors
	  groupsize 8
	  #
	  # Stop when 50% of the groups are idle
	  idleg 50
	  #
	  # List of input files start here
	  HF.crno4
	  HF_2e.crno4
	  ROHF.pyridine
	  ROHF_incore.pyridine
	  ROHF_opt.pyridine
	  UHF.morphine.6-31G-d
	  UHF_incore.pyridine
	  UHF_opt.pyridine
	  HF.Bz_crco3.TZVP
	  ROHF.Bz_crco3.TZVP
	  ECP_sbkjc.opt.crno4
	  DFT.morphine.6-31G-dp
	  DFT.morphine.6-31G-dp_harmonic
	  DFT.morphine.A2.DZVP
	  UKS.pyridine
	  DFT.siosi4.617
	  DFT.siosi5.1199
	  DFT.cyclo.6-31G
	  DFT_jfit.morphine.A2
	  DFT_jfitA.siosi5.1199
	  DFT_opt.exti4a1.3-21G
	  MP2_opt.crno4
	  MP2_ECP_opt.crno4
	  MP2_forces.scf3
	  MP2_opt.mnco5h
	  MP2_opt_props.brncs
	  RPA.pyridine
	  SECD_opt.pyridine.6-31G-dp
	  SECD.TFMtoluene.6-31G
	  SECD_ECP_opt.crco6
	  SECD_HCTH.TFMtoluene.6-31G
	  *******
	  # Anything from here on will not be processed
	  DCI.cf2.cc-pvtz
	  DCI.pyridine.tzvp