Compiling NCBI C++ Toolkit
NCBI C++ Toolkit version 12.0.0[1] is installed on Proteus. Use the module
ncbi-toolkit/gcc/64/12.0.0
There may be module dependencies -- see any warning messages that appear with "module load ncbi-toolkit".
Basic Usage
Load the Module
First, load the module which provides the toolkit:
[juser@proteusa01 ~]$ module load ncbi-toolkit/gcc/64/12.0.0
Check that it worked:
[juser@proteusa01 ~]$ which update_blastdb.pl
/mnt/HA/opt/ncbi-toolkit/gcc/64/12.0.0/bin/update_blastdb.pl
DB Location
Decide on a directory where all the database files will be downloaded.
If you are working in a group which will share the database, put this in
your group directory. We will use ~/ncbi_db/:
[juser@proteusa01 ~]$ mkdir ncbi_db
Or use the local copy. See BLAST Databases
Set the BLASTDB Environment Variable
You can set the BLASTDB environment variable[2] in one of your .bashrc
file. You can also set it manually in any job script you write.
export BLASTDB=~/ncbi_db
Download an Updated Database
We will use the nr database as an example. If the BLASTDB
environment is not set, manually set it in the shell. (See above.)
See what databases are available:
[juser@proteusa01 ~]$ update_blastdb.pl --showall
Just in case the BLASTDB environment variable is not properly used by
the tools, cd into it, and do the update -- this will take up to an
hour:
[juser@proteusa01 ~]$ cd $BLASTDB
[juser@proteusa01 ncbi_db]$ update_blastdb.pl nr
...
After it completes, check that all files were downloaded correctly by doing the checksum:
[juser@proteusa01 ncbi_db]$ md5sum -c *.md5
nr.01.tar.gz: OK
nr.02.tar.gz: OK
...
Uncompress them all:
[juser@proteusa01 ncbi_db]$ for x in nr.*.tar.gz ; do tar xf $x ; done
...
This produces many files: .phr, .psd, .psq, etc.
Delete the tarballs:
[juser@proteusa01 ncbi_db]$ rm -f *.tar.gz
Retain the *.md5 files so that the update_blastdb.pl script can tell
which db is up to date.
Run Multithreaded
The installation of NCBI Toolkit on Proteus does not use MPI, but it is
multithreaded. That means it can use multiple processor cores on a
single compute node, but will not do computations using multiple compute
nodes. Most NCBI Toolkit command line tools have the option to specify
the number of threads. In a job script, the NSLOTS environment
variable is set in the job to be the number of slots requested. So:
#$ -pe shm 8
...
blastx -num_threads ${NSLOTS} ...
WARNING
Using the NCBI-hosted databases by using the "-remote" option will get Proteus blocked by NCBI due to overuse. This is true especially for batch jobs on the cluster.
Compiling
[juser@proteusi01 ncbi_cxx--12_0_0]$ module list
Currently Loaded Modulefiles:
1) shared 2) proteus 3) gcc/4.8.1 4) sge/univa 5) hdf5_18/1.8.11
[juser@proteusi01 ncbi_cxx--12_0_0]$ ./configure LDFLAGS="-L$HDF5DIR" CPPFLAGS="-I$HDF5INCLUDE" \
--prefix=/mnt/HA/opt/ncbi_cxx/gcc/12.0.0 --with-algo --with-png --with-tiff --with-pcre \
--with-z --with-mysql --with-check --with-boost --with-xerces --with-libxslt \
--with-sge=/cm/shared/apps/sge/univa --with-xalan --with-gif --with-jpeg --with-xpm \
--with-curl --with-hdf5=${HDF5DIR} \
--with-mt --with-64 --without-debug --with-optimization --with-dll --with-runpath
=========
2014-08-28
module list
1) shared 4) sge/univa 7) proteus-fftw3/gcc/64/3.3.3 10) boost/openmpi/gcc/64/1.56.0
2) proteus 5) proteus-blas/gcc/64/20110419 8) python/2.7.8 11) hdf5_18/1.8.11
3) gcc/4.8.1 6) proteus-lapack/gcc/64/3.5.0 9) proteus-openmpi/gcc/64/1.8.1-mlnx-ofed
export CFLAGS="-O3 -mavx -msse4.2 -mfpmath=sse"
export CXXFLAGS="${CFLAGS}"
export NCBIPREFIX="/mnt/HA/opt/ncbi-toolkit/gcc/64/12.0.0
./configure --prefix=${NCBIPREFIX} --with-mt --with-64 --with-lfs --with-check \
--with-bin-release --with-strip --with-sge=$SGE_ROOT --with-3psw=std:netopt \
--with-app --with-boost=$BOOSTDIR --with-optimization --without-debug \
--with-dll
Build happens in directory GCC481-ReleaseMTDLL64.