Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Tipp zum entsprechenden Kapitel im PBSPro User´s Guide

...

Jeder Job bekommt dazu die Variable $PBS_ARRAY_INDEX gesetzt und kann dadurch unterschiedliche Programme starten.

Beim Abschicken der Jobs muss man dafür jedoch den Range für $PBS_ARRAY_INDEX mit:

Code Block
qsub -A Project -J 0-3 job.file


Beispiel 1: Parameter im Job-File

Code Block
languagebash
linenumberstrue
collapsetrue
#!/bin/bash
#PBS -l select=1:ncpus=1:mem=2gb:arch=ivybridge
#PBS -l walltime=72:00:00
#PBS -N Bachelorarbeit
#PBS -A Benchmarking
#PBS -qr testy

me=`basename $0`
LOGFILE=$PBS_O_WORKDIR/$PBS_JOBNAME"."$PBS_JOBID"_"$PBS_ARRAY_INDEX".log"

#SCRATCHDIR=/scratch_gs/$USER/$PBS_JOBID/
#mkdir -p "$SCRATCHDIR"

cd $PBS_O_WORKDIR

echo "$PBS_JOBID ($PBS_JOBNAME) @ `hostname` at `date` in "$RUNDIR" START" > $LOGFILE
echo "`date +"%d.%m.%Y-%T"`" >> $LOGFILE

echo >> $LOGFILE
echo "GLOBAL PARAMETERS" >> $LOGFILE
echo "---------------------------" >> $LOGFILE
echo "Node      : "`hostname` >> $LOGFILE
echo "RunDir    : "$PBS_O_WORKDIR >> $LOGFILE
#echo "ScratchDir: "$SCRATCHDIR >> $LOGFILE
echo "# CPUs    : "$NCPUS >> $LOGFILE
echo "# Threads : "$OMP_NUM_THREADS >> $LOGFILE

shopt -s extglob
job[1]='java -jar $PBS_O_WORKDIR/Code/elki/out/artifacts/elki_jar/elki.jar KDDCLIApplication  -evaluator NoAutomaticEvaluation -resulthandler ResultWriter -out.silentoverwrite true -dbc.in $PBS_O_WORKDIR/Datasets/half_rel_160d/half_rel_160d.csv -parser.labelIndices 161 -algorithm clustering.kmeans.KMeansLloyd -kmeans.initialization SampleKMeansInitialization -kmeans.algorithm KMeansLloyd -kmeans.samplesize 0.1 -kmeans.k 113  -dbc.filter transform.GlobalPrincipalComponentAnalysisTransform -out $PBS_O_WORKDIR/Datasets/half_rel_160d/output/pca/113'
job[2]='java -jar $PBS_O_WORKDIR/Code/elki/out/artifacts/elki_jar/elki.jar KDDCLIApplication  -evaluator NoAutomaticEvaluation -resulthandler ResultWriter -out.silentoverwrite true -dbc.in $PBS_O_WORKDIR/Datasets/half_rel_160d/half_rel_160d.csv -parser.labelIndices 161 -algorithm clustering.kmeans.KMeansLloyd -kmeans.initialization SampleKMeansInitialization -kmeans.algorithm KMeansLloyd -kmeans.samplesize 0.1 -kmeans.k 114  -dbc.filter transform.GlobalPrincipalComponentAnalysisTransform -out $PBS_O_WORKDIR/Datasets/half_rel_160d/output/pca/114'
job[3]='java -jar $PBS_O_WORKDIR/Code/elki/out/artifacts/elki_jar/elki.jar KDDCLIApplication  -evaluator NoAutomaticEvaluation -resulthandler ResultWriter -out.silentoverwrite true -dbc.in $PBS_O_WORKDIR/Datasets/half_rel_160d/half_rel_160d.csv -parser.labelIndices 161 -algorithm clustering.kmeans.KMeansLloyd -kmeans.initialization SampleKMeansInitialization -kmeans.algorithm KMeansLloyd -kmeans.samplesize 0.1 -kmeans.k 115  -dbc.filter transform.GlobalPrincipalComponentAnalysisTransform -out $PBS_O_WORKDIR/Datasets/half_rel_160d/output/pca/115'

# Select current call
run=${job[$PBS_ARRAY_INDEX]}

#cp -r $PBS_O_WORKDIR/* $SCRATCHDIR/.
#cd $SCRATCHDIR

echo >> $LOGFILE
echo "STARTING..." >> $LOGFILE
echo "---------------------------" >> $LOGFILE

eval $run

#cp -r "$SCRATCHDIR"/* $PBS_O_WORKDIR/.
cd $PBS_O_WORKDIR

echo >> $LOGFILE
qstat -f $PBS_JOBID >> $LOGFILE

echo "$PBS_JOBID ($PBS_JOBNAME) @ `hostname` at `date` in "$RUNDIR" END" >> $LOGFILE
echo "`date +"%d.%m.%Y-%T"`" >> $LOGFILE

...

Code Block
languagebash
linenumberstrue
collapsetrue
#!/bin/bash
#PBS -l select=1:ncpus=1:mem=2gb:arch=ivybridge
#PBS -l walltime=72:00:00
#PBS -N Bachelorarbeit
#PBS -A Benchmarking
#PBS -qr testy

me=`basename $0`
LOGFILE=$PBS_O_WORKDIR/$PBS_JOBNAME"."$PBS_JOBID"_"$PBS_ARRAY_INDEX".log"

#SCRATCHDIR=/scratch_gs/$USER/$PBS_JOBID/
#mkdir -p "$SCRATCHDIR"

cd $PBS_O_WORKDIR

echo "$PBS_JOBID ($PBS_JOBNAME) @ `hostname` at `date` in "$RUNDIR" START" > $LOGFILE
echo "`date +"%d.%m.%Y-%T"`" >> $LOGFILE

echo >> $LOGFILE
echo "GLOBAL PARAMETERS" >> $LOGFILE
echo "---------------------------" >> $LOGFILE
echo "Node      : "`hostname` >> $LOGFILE
echo "RunDir    : "$PBS_O_WORKDIR >> $LOGFILE
#echo "ScratchDir: "$SCRATCHDIR >> $LOGFILE
echo "# CPUs    : "$NCPUS >> $LOGFILE
echo "# Threads : "$OMP_NUM_THREADS >> $LOGFILE

shopt -s extglob

parameterFile="parameter_per_job.input"

# Select current call
run=$(sed "${$PBS_ARRAY_INDEX}q;d" $parameterFile}

#cp -r $PBS_O_WORKDIR/* $SCRATCHDIR/.
#cd $SCRATCHDIR

echo >> $LOGFILE
echo "STARTING..." >> $LOGFILE
echo "---------------------------" >> $LOGFILE

eval $run

#cp -r "$SCRATCHDIR"/* $PBS_O_WORKDIR/.
cd $PBS_O_WORKDIR

echo >> $LOGFILE
qstat -f $PBS_JOBID >> $LOGFILE

echo "$PBS_JOBID ($PBS_JOBNAME) @ `hostname` at `date` in "$RUNDIR" END" >> $LOGFILE
echo "`date +"%d.%m.%Y-%T"`" >> $LOGFILE

...

Code Block
languagebash
linenumberstrue
java -jar $PBS_O_WORKDIR/Code/elki/out/artifacts/elki_jar/elki.jar KDDCLIApplication  -evaluator NoAutomaticEvaluation -resulthandler ResultWriter -out.silentoverwrite true -dbc.in $PBS_O_WORKDIR/Datasets/half_rel_160d/half_rel_160d.csv -parser.labelIndices 161 -algorithm clustering.kmeans.KMeansLloyd -kmeans.initialization SampleKMeansInitialization -kmeans.algorithm KMeansLloyd -kmeans.samplesize 0.1 -kmeans.k 113  -dbc.filter transform.GlobalPrincipalComponentAnalysisTransform -out $PBS_O_WORKDIR/Datasets/half_rel_160d/output/pca/113
java -jar $PBS_O_WORKDIR/Code/elki/out/artifacts/elki_jar/elki.jar KDDCLIApplication  -evaluator NoAutomaticEvaluation -resulthandler ResultWriter -out.silentoverwrite true -dbc.in $PBS_O_WORKDIR/Datasets/half_rel_160d/half_rel_160d.csv -parser.labelIndices 161 -algorithm clustering.kmeans.KMeansLloyd -kmeans.initialization SampleKMeansInitialization -kmeans.algorithm KMeansLloyd -kmeans.samplesize 0.1 -kmeans.k 114  -dbc.filter transform.GlobalPrincipalComponentAnalysisTransform -out $PBS_O_WORKDIR/Datasets/half_rel_160d/output/pca/114
java -jar $PBS_O_WORKDIR/Code/elki/out/artifacts/elki_jar/elki.jar KDDCLIApplication  -evaluator NoAutomaticEvaluation -resulthandler ResultWriter -out.silentoverwrite true -dbc.in $PBS_O_WORKDIR/Datasets/half_rel_160d/half_rel_160d.csv -parser.labelIndices 161 -algorithm clustering.kmeans.KMeansLloyd -kmeans.initialization SampleKMeansInitialization -kmeans.algorithm KMeansLloyd -kmeans.samplesize 0.1 -kmeans.k 115  -dbc.filter transform.GlobalPrincipalComponentAnalysisTransform -out $PBS_O_WORKDIR/Datasets/half_rel_160d/output/pca/115


Tipp: Für weitere Details siehe auch User Manuals → PBSPro User´s Guide → Kapitel 8 "Job Arrays".