Software Products

SDSS makes use of many software products in its operations, data reduction, and data analysis pipelines, most of which are developed by members of the collaboration. Along with data products, these software products are organised in repositories that are version controlled, tagged, and also released publicly during each data release. Here we describe the various ways of accessing, downloading, and installing public SDSS software products.

Publicly Available Data and Software

SDSS provides public access to many data and software products. Historically these products were hosted via SVN. SDSS has begun a process of migrating many of its products to git, hosted them on Github, with some Python products being packaged and deployed on the Python Package Index (PyPI)

SDSS on PyPI

PyPI allows for easily deployable Python packages installable with

pip install [package_name]

Some SDSS packages are now available on PyPI. See the SDSS PyPI page for an official list of SDSS packages.

SDSS on Github

SDSS is now on Github. See which packages are available on the SDSS Organization. All new SDSS products will be made available only on Github. Older products are being migrated over from SVN. See the Github Cheatsheet to Git for a handy guide of git commands. Example syntax to check out a Github product

git clone https://github.com/sdss/[product].git

Software Docs Hosted on ReadtheDocs

Some SDSS software packages on PyPI or GitHub have more detailed documentation specific to the package hosted on ReadtheDocs.

- sdss-tree: Sets up a local SDSS environment
- sdss-access: Access and Download SDSS data.
- sdss-marvin: Interact with MaNGA data.
- sdss-python-template: Create new SDSS Python projects.
- sdssdb: SDSS Database Management
- sdss-brain: SDSS Helpers for Multi-Modal Data Access

SDSS on SVN

Many older SDSS data and software products exist on SVN, viewable in as a full browser tree. SDSS SVN products fall in one of three categories:

data
Repositories containing large datasets, e.g. catalog and metadata files used by SDSS pipelines
repo
Repositories containing software code and tools
deprecated
Repositories from older SDSS surveys that have been deprecated, and are no longer actively maintained or supported.

See Downloading SVN Products below for details on how to access the available SDSS products.

Downloading SVN Products

Generally speaking, subversion (a.k.a. SVN) products are organized into three subdirectories.

trunk
holds the very latest development version of the software. Stability is not guaranteed, but if you need to track developments in 'real time', you should check out trunk.
tags
holds labeled versions of the software that are intended for release.
branches
holds active development areas that are meant to be kept separate from trunk. Only experts should look here.

If you only want a copy of the software, a subversion export is the best option. For example to download version v5_5_33 of idlutils:

svn export https://svn.sdss.org/public/repo/sdss/idlutils/tags/v5_5_33 idlutils-v5_5_33

This will copy the contents of the repository tag to the local directory idlutils-v5_5_33. Checking out the trunk of a product (if you really need to do this) is somewhat different:

svn co https://svn.sdss.org/public/repo/sdss/idlutils/trunk idlutils-trunk

Your checkout of the trunk can be kept up-to-date by doing 'svn up'. Consult the subversion documentation for further details.

Read-Write Access

Read-write access (being able to commit to a product in addition to checking it out) is limited to members of the SDSS collaboration. If you are a member of the collaboration, you can set up read-write access by following the instructions on the SDSS wiki (and you will know how to find this).

Installing SDSS Software

While many SDSS products can be installed directly from PyPI, GitHub, or SVN, SDSS provides a tool to simplify the installation of its products.

If you use TCLSH modules or LUA modules (LMOD), then sdss_install provides an automated way to install SDSS software and data products. The procedure below describes how to bootstrap sdss_install, which will then allow you to sdss_install other products.

sdss_install downloads the software, builds it and adds a module file to setup the required environmental variables (including $PATH, etc.)

  1. Install TCLSH modules or LUA modules (LMOD)
  2. Create a new directory under which all of your sdss_install’ed software will be built, and associated modulefiles directories. For example: 
    mkdir -p ~/software/sdss/github/modulefiles
mkdir -p ~/software/sdss/svn/modulefiles
cd ~/software/sdss
  3. Set SDSS_INSTALL_PRODUCT_ROOT to the directory you created in the previous step, following that example:
    For bash

    export SDSS_INSTALL_PRODUCT_ROOT=~/software/sdss

    For csh

    setenv SDSS_INSTALL_PRODUCT_ROOT ~/software/sdss
  4. Add the sdss_install modulefiles directories to your module path, following the example from the previous 2 steps: 
    module use ~/software/sdss/github/modulefiles
module use ~/software/sdss/svn/modulefiles
  5. Clone sdss_install from SDSS GitHub: 
    git clone https://github.com/sdss/sdss_install.git $SDSS_INSTALL_PRODUCT_ROOT/github/sdss_install/master
  6. Run the bootstrap installer: 
    $SDSS_INSTALL_PRODUCT_ROOT/github/sdss_install/master/bin/sdss_install_bootstrap
  7. Get sdss_install into you path: 
    module load sdss_install
  8. Steps 4 and 7 may be added to your ~/.modules file, and that file sourced by your .bashrc or .tcshrc for convenience.

It will now be possible to install SDSS-IV (and in future SDSS-V) data or software products by specifying the partial path to the product, e.g.:

sdss_install --public sdss/firefly v1_1_0

If you use TCLSH modules or LUA modules (LMOD), then sdss4tools provides an automated way to install SDSS software and data products. The procedure below describes how to bootstrap sdss4tools, which will then allow you to sdss4install other products.

sdss4install downloads the software, builds it and adds a module file to setup the required environmental variables (including $PATH, etc.)

  1. Install TCLSH modules or LUA modules (LMOD)
  2. Create a new directory under which all of your sdss4installed software will be built, for example:
    mkdir ~/software/sdss
    cd ~/software/sdss
  3. Set SDSS4_PRODUCT_ROOT to the directory you created in the previous step, following that example:
    for bash
    export SDSS4_PRODUCT_ROOT=~/software/sdss
    for csh
    setenv SDSS4_PRODUCT_ROOT=/software/sdss
  4. Add the sdss4 modulefiles directory to your module path, following the example from the previous 2 steps:
    module use ~/software/sdss/modulefiles
  5. Download the bootstrap installer:
    svn export https://svn.sdss.org/public/repo/sdss/sdss4tools/trunk/bin/sdss4bootstrap
  6. Run the bootstrap installer:
    ./sdss4bootstrap
  7. Get sdss4tools into you path:
    module load sdss4tools
  8. Steps 4 and 7 may be added to your ~/.module file, and that file sourced by your .bashrc or .tcshrc for convenience.

It will now be possible to install SDSS-IV data or software products by specifying the partial path to the product, e.g.:

sdss4install --public data/sdss/catalogs/dust v0_2

sdss4install --public sdss/idlutils v5_5_17

If you use EUPS at your institution, then sdss3tools is a more automated way to install SDSS-III archival software. The procedure below describes how to install sdss3tools, which will then allow you to install other products.

If you don't know what EUPS is, you should ignore this section.

  1. Install EUPS
  2. Download the bootstrap installer:
    svn export https://svn.sdss.org/public/sdss3/repo/sdss3tools/trunk/bin/sdss3bootstrap
  3. Run the bootstrap installer:
    /bin/bash sdss3bootstrap
  4. Get sdss3tools into you path:
    setup sdss3tools

It will now be possible to install SDSS-III products by doing, e.g.:

sdss3install -c idlutils v5_5_14
Back to Top