IBM ILOG CPLEX Optimization Studio Solver: Configuration, Setup, Log, Tuning

Running IBM ILOG CPLEX Optimization Studio on HSUper

• Login on HSUper, e.g., via MobaXterm (only on Windows), PuTTY or via the ssh command.
• Use a SLURM script to submit jobs on HSUper via entering in the terminal sbatch (followed by the name of the SLURM script).
• Enter squeue -u $USER to see all your jobs in the queue (or replace $USER by an username); R at ST means that your job is currently running.
• Enter squeue --format="%.18i %.9P %.30j %.8u %.8T %.10M %.9l %.6D %R" --me to extend the length of the output of squeue.
• Enter scancel -u $USER to scancel all your jobs or scancel followed by a jobid to cancel the corresponding job.
• Enter sacct -S now-7days -X --user=$USER --format="jobid,jobname,user,account,partition,AllocCPUS,AllocNodes,State,ConsumedEnergy,Elapsed,TimeLimit,ExitCode" to gain information about all your jobs during the last 7 days.
• The oplrun command enables you to execute an OPL model or project from the Windows command line or in a UNIX environment.
• A basic SLURM script could look like this:

#!/bin/bash

#SBATCH --job-name=name       ### the name of your job
#SBATCH --output=%x_%j.out    ### the output file for errors etc.

#SBATCH --nodes=1             ### number of nodes
#SBATCH --ntasks=1            ### number of (MPI) tasks needed
#SBATCH --partition=small     ### partition (small, small_shared, small_fat, dev)
#SBATCH --cpus-per-task=72    ### number of threads per task (OMP threads)
#SBATCH --time=24:00:00       ### maximum wall clock limit for job execution

oplrun myModel.mod myData.dat > log.txt

• If you want to solve several problems, e.g., with different .mod or .dat files or with varying numbers of threads (see _How to change parameters in oplrun), you can add these runs in the same SLURM script in a new line after oplrun myModel.mod myData.dat > log.txt such that these problems are solved one after the other.
  If you want to start all your runs as soon as possible, then you can use individual SLURM scripts and submit each of them via sbatch (followed by the name of the corresponding SLURM script). Or you could use a bash script (chmod +x myBash makes the bash script myBash executable, ./myBash executes it) to start the different SLURM scripts. The following bash script starts runs for 30 different problems (each problem is indentical except that a different data file is used with j = 1, …, 30 with start p=1 and end P=30) and different numbers of threads (i.e., i = 1, 2, 4, 8, 16, 32, 64 with start t=1 and end T=72).

#!/bin/bash

t=1
T=72
p=1
P=30
for (( j=$p ; j<=$P ; j++ ));
do
  for (( i=$t ; i<=$T ; i*=2 ));
  do
    sbatch --job-name="threads$i.problem$j" --export=ALL,i="$i",j="$j" var_param_oplrun.slurm
  done
done

The SLURM script var_param_oplrun.slurm is identical to the SLURM script from above, except that the line
oplrun myModel.mod myData.dat > log.txt
is changed to
oplrun -D myThreads=$i myModel.mod myData$j.dat > log$i.$j.txt.

Exporting MPS files from IBM ILOG CPLEX Optimization Studio

• The MPS format has not been standardized, and each solver may have its own set of conventions and extensions.
• CPLEX command line:
  • write myModel.mps
  • Compress it directly if you append .gz after the filename, i.e. myModel.mps.gz_.
  • If you want to anonymize the variable and constraint names append the file ending .rew instead of .mps.
• oplrun: oplrun -e myModel.mps myModel.mod myData.dat
• References:
  • Exporting MPS files from open-source and commercial solvers
  • How can I export a .mps file from my .mod and a .dat files in CPLEX?

How to change parameters in oplrun?

• Add, e.g., the following to your myModel.mod file:

int myThreads = ...;
int myParallelmode = ...;

execute
{
  cplex.threads = myThreads;
  cplex.parallelmode = myParallelmode;
}

• Then, call oplrun via: oplrun myModel.mod myData.dat -D myThreads=4 -D myParallelmode=1 to run oplrun with 4 threads (default setting is 0 (zero) or equivalently without setting this parameter which leads to min(32, number of cores of the machine) threads, i.e., 32 threads on HSUper) in deterministic mode.
• Call oplrun via: oplrun -v -profile myModel.mod myData.dat to run oplrun in verbose mode and output in addition profiler information.
• The option -tune enables parameter fine-tuning.
• References:
  • How to change CPLEX parameters in oplrun?
  • global thread count
  • parallel mode switch
  • Command line options