Q-Chem on HPC
Q-Chem is a software package for analyzing “molecular structures, reactivities, and vibrational, electronic, and NMR spectra” that is available to HPC researchers by permission. To request access to this software on HPC, please email firstname.lastname@example.org
NOTE: Due to licensing restrictions with Gaussian, HPC users may not use Q-Chem and Guassian concurrently. Please email email@example.com to switch access.
This page will help you set up Q-Chem on your HPC account.
Setting Up Your Q-Chem Environment
Q-Chem requires a number of environment variables to run properly on HPC. Some environment settings are configured automatically when you run Q-Chem’s HPC setup script:
$ source /usr/usc/qchem/5.1/setup.sh $ env | grep QC QC=/auto/usc/qchem/5.1 QCPROG_S=/auto/usc/qchem/5.1/exe/qcprog.exe_s QCRSH=ssh QCPLATFORM=LINUX_Ix86_64 QCAUX=/auto/usc/qchem/5.1/qcaux QCPROG=/auto/usc/qchem/5.1/exe/qcprog.exe QCMPI=mpich
Other variables that you will need to configure include the scratch directory, the machine file, and a location for your temporary files. We also recommend that you configure your Q-Chem job for checkpointing.
Q-Chem requires that you set the environment variable QCSCRATCH. You should set it to the directory that Q-Chem will use for storing temporary files when running a job.
One suggestion is to set QCSCRATCH to $SCRATCHDIR. SCRATCHDIR is the temporary directory that Slurm creates for its own temporary job files.
Q-Chem also requires that you set the environment variable QCMACHINEFILE. This should be set to a text file that contains a list of all hostnames that your job has access to.
This information is only relevant for the duration of the job, so we suggest you place the file in $QCSCRATCH.
You can create machinefile by redirecting the output of the Slurm command scontrol show hostname to $QCMACHINEFILE which will display the machines being used for the current job.
scontrol show hostname > machinefile
Each Q-Chem job may have different requirements for the machine file; it may be necessary to modify the output of the scontrol command depending on your job’s requirements.
Saving Temporary Files
If you wish to save KEY temporary files produced by Q-Chem during a run, you can run the program with a save directory argument. By default, Q-Chem will save the files to $QCSCRATCH/savedir.
NOTE: Slurm’s $SCRATCHDIR is deleted at the end of each job. If you have set QCSCRATCH to $SCRATCHDIR, you MUST copy the files to a persistent location before the end of your job.
export QCSCRATCH=$SCRATCHDIR qchem data.in data.out data.savedir cp -r $QCSCRATCH/data.savedir .
Alternatively, you can set QCSCRATCH to a persistent location. Do this if you want to save ALL temporary files (normally not required) using the qchem –save option.
export QCSCRATCH=/staging/tt1/ttrojan qchem -save data.in data.out data.savedir
HPC recommends checkpointing your Q-Chem job, especially if you plan to use the scavenge partition. A Q-Chem checkpoint file can be requested by setting GUI = 2 in the $rem section of the input file. The checkpoint file name is determined by the $GUIFILE environment variable.
Running a Q-Chem Job
The most basic way to run Q-Chem on a single processor (default) would be to run a command in the following form:
qchem data.in data.out data.savedir &
where data.in is the name of your Q-Chem input file, data.out is the name of the file where Q-Chem will place its output, and data.dir is the directory where Q-Chem will save key temporary files.
NOTE: If an output file already exists, it will be overwritten.
The & places the job into the background so that you may continue to work in the current shell.
See the Running Q-Chem manual page on the Q-Chem website for more details.
To run Q-Chem in parallel, the number of the number of threads, nt, (assumes 1 thread=1 CPU core) and the number of MPI processes (CPUs), np, may need be specified. If np is not specified, Q-Chem will default to running locally on a single node.
The default command, above, is equivalent to:
qchem -nt 1 data.in data.out data.savedir &
When -np is not given, Q-Chem will default to running locally on a single node.
Q-Chem supports multiple methods to run in parallel, as described in the table below.
|OpenMP||Multiple threads (CPUs)||Single task
(Multiple CPUs per task)
|MPI||Multiple MPI processes
(One CPU each)
(One CPU per task)
|MPI+OpenMP (Hybrid)||Multiple MPI processes
(Multiple CPUs each)
(Multiple CPUs per task)
Example Q-Chem Script
There are a number of sample Q-Chem jobs available in /usr/usc/qchem/<version>/samples. You can use these sample input files as a starting place to building your own files.
To use a sample directory, copy it to your project directory and run it there. Compare your output with the reference files (.out) of the same name.
cp -r /auto/usc/qchem/5.1/samples/freq .
Below is a template script that can be used to run Q-Chem on either a single- or multi-node job using a sample input file.
#!/bin/bash #SBATCH --export=none #SBATCH --ntasks=2 #SBATCH --cpus-per-task=12 #SBATCH --time=1:00:00 #SBATCH --mem-per-cpu=2GB #SBATCH --job-name=qchem source /usr/usc/qchem/5.1/setup.sh # Define the scratch dir for savedir and machine file export QCSCRATCH=$SCRATCHDIR export QCMACHINEFILE=$SCRATCHDIR/machinefile # Create machine file for this job scontrol show hostname > $QCMACHINEFILE # Sample files were copied to this directory cd $SLURM_SUBMIT_DIR/freq # $SLURM_NTASKS will take the value provided by --ntasks option # $SLURM_CPUS_PER_TASK will take the value provided by --cpus-per-task option qchem -np $SLURM_NTASKS -nt $SLURM_CPUS_PER_TASK FREQ_water.in FREQ_water.out FREQ_water # Save key temporary files for curiosity cp -r $QCSCRATCH/FREQ_water .