Background

This page provides updates for the AMBER 10 software to enable acceleration on NVIDIA GPUs.

Support is currently EXPERIMENTAL and PROVISIONAL. You use this software at your own risk. These patches are designed for EXPERTS ONLY, if you are NOT an expert in using AMBER, comfortable with extensively validating AMBER executables and ensuring that the results generated are correct then you should NOT use these patches. Additionally it is expected that you are familiar with AMBER compilation and the use of NVIDIA GPUs including CUDA support within a Linux environment.

It is assumed that you already have the NVIDIA CUDA drivers, CUDA Toolset and CUDA SDK installed and correctly configured. For more info refer to http://www.nvidia.com/cuda/.

Authorship & Support

Patch Author: Ross C. Walker (SDSC)

NVIDIA CUDA Implementation:

Scott Le Grand (NVIDIA)
Duncan Poole (NVIDIA)
Mark Williamson (SDSC)
Ross Walker (SDSC)

Please note that this patch is EXPERIMENTAL! All questions or comments should be directed to Ross Walker via the AMBER mailing list (http://lists.ambermd.org/) and will be answered on a voluntary basis as and when time is available. Feedback is welcome please just realize that this implementation is to be considered ALPHA software at present. Validation of the GPU implementation has been undertaken but considerable work is still required to ensure that ensemble averages and measurable properties match the cpu implementations. Initial tests show that energy conservation is equivalent, however, other issues may remain. Bugs are also probably present. Hence caveat emptor. You use this code at your own risk.

Funding for this work has been graciously provided by NVIDIA and The National Science Foundation's TeraGrid Advanced User Support Program through the San Diego Supercomputer Center.

Supported Features (v0.1)

Currently as of version 0.1 support is provided for Generalized Born implicit solvent simulations within the AMBER PMEMD software. Support for PME simulations is planned but not currently supported as of this patch. The following options are not supported:

Additional Notes: Currently multiple GPUs are NOT supported. If you have multiple CUDA capable GPUs in your system then the one with the most memory will be used.

Supported GPUS

The code currently supports any NVIDIA GPU supporting feature set > 1.3 or better. As of Sept 3rd 2009 this includes the following GPUs:

Tesla C1060
Tesla C1070*
GTX295
GTX285*
GTX275*

* = untested.

Provisional Benchmarks (v0.1)

The following tables provide some provisional benchmarks for various system sizes.

Machine Specs
CPU = Dual x Quad Core Intel E5462 2.80 GHz
MPICH2.0 - 1.07
MKL 10.1.1.019
Ifort 10.1.018

GPU = Tesla C1060
GPU = Tesla GTX295
Code Base = AMBER 10 Release / AMBER 10 + CUDA Patch v0.1

1) TRPCage = 304 atoms

&cntrl
  imin=0,irest=1,ntx=5,
  nstlim=100000,dt=0.002,ntb=0,
  ntf=2,ntc=2,tol=0.000001,
  ntpr=1000, ntwx=1000, ntwr=50000,
  cut=9999.0, rgbmax=15.0,
  igb=1,ntt=0,nscm=0,
/

2) Myoglobin = 2492 atoms

&cntrl
  imin=0,irest=1,ntx=5,
  nstlim=10000,dt=0.002,ntb=0,
  ntf=2,ntc=2,tol=0.000001,
  ntpr=1000, ntwx=1000, ntwr=50000,
  cut=9999.0, rgbmax=15.0,
  igb=1,ntt=0,nscm=0,
/

3) Nucleosome = 25095 atoms

&cntrl
  imin=0,irest=1,ntx=5,
  nstlim=1000,dt=0.002,ntb=0,
  ntf=2,ntc=2,tol=0.000001,
  ntpr=100, ntwx=100, ntwr=50000,
  cut=9999.0, rgbmax=15.0,
  igb=1,ntt=0,nscm=0,
/

Download & Installation Instructions

The provisional GPU support is designed to be applied to AMBER 10 in the form of a tar.gz file AND a patch file.

These instructions assume that you have the CUDA drivers, toolset and SDK correctly installed on your machine.

1) Download the following two files and save them in a path of your choosing:

i) amber10_nvidia_cuda_support_v0.1.patch
ii) amber10_nvidia_cuda_support_v0.1.tar.gz

2) Create a CLEAN AMBER 10 installation for yourself.

cd new_amber10_path
tar xvjf /path_to_ambertools_file/AmberTools-1.2.tar.bz2
tar xvjf /path_to_amber10_file/Amber10.tar.bz2
cd amber10
export AMBERHOME=/path_to_amber10/
wget http://ambermd.org/bugfixes/10.0/bugfix.all
patch -p0 < bugfix.all
wget http://ambermd.org/bugfixes/AmberTools/1.2/bugfix.all
patch -p0 < bugfix.all

3) Apply the NVIDIA CUDA Patch

tar xvzf amber10_nvidia_cuda_support_v0.1.tar.gz
patch -p0 < amber10_nvidia_cude_support_v0.1.patch

4) Compile the pmemd.cuda executable

cd $AMBERHOME/src/pmemd/
./configure linux_em64t_cuda gfortran nopar pubfft
make
make install

5) Test pmemd.cuda is functioning correctly.

cd $AMBERHOME/test/
make test.pmemd.cuda

All test cases should pass.

You can now run using the GPU by using the pmemd.cuda executable in place of pmemd.