Job Script Example 01 Many Input Files

Description

You have many input files to process: maybe you are converting data from one format to another. The file names are in sequential order:

rawdata01.txt rawdata02.txt ... rawdata13.txt

The example script plus input files are in

/mnt/HA/opt/Examples/Example01

Script

#!/bin/bash
#$ -S /bin/bash
#$ -N example01
#$ -j y
#$ -cwd
#$ -M fixme@drexel.edu
#$ -P fixmePrj
#$ -l h_rt=300
#$ -q all.q@@amdhosts
#$ -t 1:13:1

. /etc/profile
module load shared
module load sge/univa

DATADIR=/scratch/$USER/Example01

### This makes the fileid an integer with zeroes in front so that all fileids are the same number of characters
fileid=$(printf %02d $SGE_TASK_ID)

sed -e 's/hello/goodbye/' ${DATADIR}/rawdata${fileid}.txt > ${DATADIR}/moddata${fileid}.txt