Software Installation on Pegasus

Pegasus users are free to compile and install software in their own home directories, by following the software’s source code or local installation instructions.

To install personal software on the Pegasus cluster, navigate to an interactive node by submitting an interactive shell job to the Pegasus cluster LSF scheduler. More on Pegasus interactive jobs.

Source code software installations (“compilations”) can only be performed in your local directories. Users of Pegasus are not administrators of the cluster, and therefore cannot install software with the sudo command (or with package managers like yum / apt-get). If the software publisher does not provide compilation instructions, look for non-standard location installation instructions.

In general, local software installation involves:

confirming pre-requisite software & library availability, versions
downloading and extracting files
configuring the installation prefix to a local directory (compile only)
compiling the software (compile only)
updating PATH and creating symbolic links (optional)

Confirm that your software’s pre-requisites are met, either in your local environment or on Pegasus as a module. You will need to load any Pegasus modules that are pre-requisites and install locally any other pre-requisites.

We suggest keeping downloaded source files separate from compiled files (and any downloaded binary files).

ACS does not install user software. Request cluster software installations from hpc@ccs.miami.edu

Downloading and extracting files

If necessary, create software directories under your home directory:

[username@pegasus ~]$ mkdir ~/local ~/src

We suggest keeping your compiled software separate from any downloaded files. Consider keeping downloaded binaries (pre-compiled software) separate from source files if you will be installing many different programs. These directories do not need to be named exactly as shown above.

Navigate to the `src` directory and download files:

Some programs require configuration and compilation (like autoconf). Other programs are pre-compiled and simply need to be extracted (like Firefox). Read and follow all instructions provided for each program.

[username@pegasus ~]$ cd ~/src
[username@pegasus src]$ wget http://ftp.gnu.org/gnu/autoconf/autoconf-2.69.tar.gz
[username@pegasus src]$ wget http://ftp.mozilla.org/pub/mozilla.org/firefox/releases/36.0/linux-x86_64/en-GB/firefox-36.0.tar.bz2

Pre-compiled software can be extracted and immediately moved into your local software directory. We suggest maintaining subdirectories with application names and version numbers, as shown below.

Extract downloaded contents:

For pre-compiled software, extract and move contents to your local software directory. For software that must be configured and compiled, extract and move contents to your source files directory.

Extraction flags:

tar.gz xvzf eXtract, Verbose, filter through gZip, using File
tar.bz2 xvjf …filter through bzip2 (j)

Extract pre-compiled software and move to local software directory:

[username@pegasus src]$ tar xvjf firefox-36.0.tar.bz2
[username@pegasus src]$ mv firefox-36.0 $HOME/local/firefox/36

The newly-extracted Firefox executable should now be located in ~/local/firefox/36/firefox Pre-compiled binaries, skip to Updating PATH and creating symbolic links.

Extract source code and `cd` to new directory:

[username@pegasus src]$ tar xvzf autoconf-2.69.tar.gz
[username@pegasus src]$ cd autoconf-2.69
[username@pegasus autoconf-2.69]$

Source code, proceed to *Configuring installation and compiling software*.

Configuring installation and compilation

We suggest using subdirectories with application names and version numbers, as shown below. There may be other configuration settings specific to your software.

Configure with local directory prefix (absolute path):

Configuration files may also be located in the bin (binary) directory, usually software/bin

[username@pegasus autoconf-2.69]$ ./configure --prefix=$HOME/local/autoconf/2.69

Make and install the software:

[username@pegasus autoconf-2.69]$ make
[username@pegasus autoconf-2.69]$ make install
...

If there are dependencies or conflicts, investigate the error output and try to resolve each error individually (install missing dependencies, check for specific flags suggested by software authors, check your local variables).

Updating PATH

PATH directories are searched in order. To ensure your compiled or downloaded software is found and used first, prepend the software executable location (usually in software/bin or software directories) to your PATH environment variable. Remember to add :$PATH to preserve existing environment variables.

Prepend software location to your `PATH` environment variable:

[username@pegasus ~]$ export PATH=$HOME/local/autoconf/2.69/bin:$PATH

Confirm by checking `which` software:

[username@pegasus ~]$ which autoconf
~/local/autoconf/2.69/bin/autoconf

Check software version:

Version flags may be software-dependent. Some common flags include --version, -v, and -V.

[username@pegasus ~]$ autoconf --version
autoconf (GNU Autoconf) 2.69
...

Create symbolic links

To maintain multiple different versions of a program, use soft symbolic links to differentiate between the installation locations. Make sure the link and the directory names are distinct (example below). If local software has been kept in subdirectories with application names and version numbers, symlinks are not likely to conflict with other files or directories.

Create a distinctly-named symlink:

This symbolic link should point to the local software executable. The first argument is the local software executable location (~/local/firefox/36/firefox). The second argument is the symlink name and location (~/local/firefox36).

[username@pegasus ~]$ ln -s ~/local/firefox/36/firefox ~/local/firefox36

Append the local location to your `PATH` environment variable:

Remember to add :$PATH to preserve existing environment variables.

[username@pegasus ~]$ export PATH=$PATH:$HOME/local

Confirm both cluster copy and recently installed software:

The cluster copy of Firefox is firefox. The recently installed local copy is firefox36 from the symbolic links created above.

[username@pegasus ~]$ which firefox
/usr/bin/firefox
[username@pegasus ~]$ firefox --version
Mozilla Firefox 17.0.10

[username@pegasus ~]$ which firefox36
~/local/firefox36
[username@pegasus ~]$ firefox36 --version
Mozilla Firefox 36.0

Reminder - to launch Firefox, connect to Pegasus via SSH with X11 forwarding enabled.

Persistent `PATH`

To persist additions to your PATH variable, edit the appropriate profile configuration file in your home directory. For Bash on Pegasus, this is .bash_profile.

Update `PATH` in shell configuration (bash):

Use echo and the append redirect (>>) to update PATH in .bash_profile.

[username@pegasus ~]$ echo 'export PATH=$HOME/local/autoconf/2.69/bin:$PATH' >> ~/.bash_profile
[username@pegasus ~]$ echo 'export PATH=$PATH:$HOME/local' >> ~/.bash_profile

both in one command (note the newline special character **``n``* directly in between the commands:*

[username@pegasus ~]$ echo -e 'export PATH=$HOME/local/autoconf/2.69/bin:$PATH\nexport PATH=$PATH:$HOME/local' >> ~/.bash_profile

or edit the file directly:

[username@pegasus ~]$ vi ~/.bash_profile
...
PATH=$PATH:$HOME/bin
PATH=$HOME/local/autoconf/2.69/bin:$PATH
PATH=$PATH:$HOME/local
...

Reload shell configurations (Bash) and check `PATH`:

Look for the recently added path locations and their order.

[username@pegasus ~]$ source ~/.bash_profile
[username@pegasus ~]$ echo $PATH
/nethome/username/local/autoconf/2.69/bin:/share/opt/python/2.7.3/bin: ... :/share/sys65/root/sbin:/nethome/username/bin:/nethome/username/local