Software Products
SDSS uses many data and software repositories in its operations, data reduction, and data analysis pipelines. Along with data products, these repository products are version controlled, tagged, and also released publicly during each data release. Here we describe the various ways of accessing, downloading, and installing public SDSS software products.
Publicly Available Data and Software
SDSS on PyPI
pip install [package_name]
Some SDSS packages are now available on PyPI. See the SDSS PyPI page for an official list of SDSS packages.
SDSS on Github
git clone https://github.com/sdss/[product].git
Software Docs Hosted on ReadtheDocs
- sdss-tree: Sets up a local SDSS environment
- sdss-access: Access and Download SDSS data.
- sdss-marvin: Interact with MaNGA data.
- sdss-python-template: Create new SDSS Python projects.
- sdssdb: SDSS Database Management
SDSS on SVN
- data
- Repositories containing large datasets, e.g. catalog and metadata files used by SDSS pipelines
- repo
- Repositories containing software code and tools
- deprecated
- Repositories from older SDSS surveys that have been deprecated, and are no longer actively used/supported/maintained.
See Downloading SVN Products below for details on how to access the available SDSS products.
Downloading SVN Products
- trunk
- holds the very latest development version of the software. Stability is not guaranteed, but if you need to track developments in 'real time', you should check out trunk.
- tags
- holds labeled versions of the software that are intended for release.
- branches
- holds active development areas that are meant to be kept separate from trunk. Only experts should look here.
If you only want a copy of the software, a subversion export is the best option. For example to download version v5_5_33 of idlutils:
svn export https://svn.sdss.org/public/repo/sdss/idlutils/tags/v5_5_33 idlutils-v5_5_33
Checking out the trunk of a product (if you really need to do this) is somewhat different:
svn co https://svn.sdss.org/public/repo/sdss/idlutils/trunk idlutils-trunk
Your checkout of the trunk can be kept up-to-date by doing 'svn up'. Consult the subversion documentation for further details.
Read-Write Access
Read-write access is limited to members of the SDSS collaboration. If you are a member of the collaboration, you can set up read-write access by following the instructions on the SDSS wiki (and you will know how to find this).
Installing SDSS Software
While many SDSS products can be installed directly from Github or SVN, SDSS provides a tool to simplify the installation of its products.
If you use TCLSH modules or LUA modules (LMOD), then sdss_install provides an automated way to install SDSS software and data products. The procedure below describes how to bootstrap sdss_install, which will then allow you to sdss_install other products.
sdss_install downloads the software, builds it and adds a module file to setup the required environmental variables (including $PATH, etc.)
- Install TCLSH modules or LUA modules (LMOD)
- Create a new directory under which all of your sdss_install’ed software will be built, and associated modulefiles directories. For example:
mkdir -p ~/software/sdss/github/modulefiles
mkdir -p ~/software/sdss/svn/modulefiles
cd ~/software/sdss - Set SDSS_INSTALL_PRODUCT_ROOT to the directory you created in the previous step, following that example:
for bash
export SDSS_INSTALL_PRODUCT_ROOT=~/software/sdss
for csh
setenv SDSS_INSTALL_PRODUCT_ROOT /software/sdss
- Add the sdss_install modulefiles directories to your module path, following the example from the previous 2 steps:
module use ~/software/sdss/github/modulefiles
module use ~/software/sdss/svn/modulefiles - Clone sdss_install from SDSS GitHub:
git clone https://github.com/sdss/sdss_install.git $SDSS_INSTALL_PRODUCT_ROOT/github/sdss_install/master
- Run the bootstrap installer:
$SDSS_INSTALL_PRODUCT_ROOT/github/sdss_install/master/bin/sdss_install_bootstrap
- Get sdss_install into you path:
module load sdss_install
- Steps 4 and 7 may be added to your ~/.modules file, and that file sourced by your .bashrc or .tcshrc for convenience.
It will now be possible to install SDSS-IV (and in future SDSS-V) data or software products by specifying the partial path to the product, e.g.:
sdss_install --github marvin 2.2.6 sdss_install --public sdss/firefly v1_1_0
If you use TCLSH modules or LUA modules (LMOD), then sdss4tools provides an automated way to install SDSS software and data products. The procedure below describes how to bootstrap sdss4tools, which will then allow you to sdss4install other products.
sdss4install downloads the software, builds it and adds a module file to setup the required environmental variables (including $PATH, etc.)
- Install TCLSH modules or LUA modules (LMOD)
- Create a new directory under which all of your sdss4installed software will be built, for example:
mkdir ~/software/sdss
cd ~/software/sdss - Set SDSS4_PRODUCT_ROOT to the directory you created in the previous step, following that example:
for bash
export SDSS4_PRODUCT_ROOT=~/software/sdss
for csh
setenv SDSS4_PRODUCT_ROOT=/software/sdss
- Add the sdss4 modulefiles directory to your module path, following the example from the previous 2 steps:
module use ~/software/sdss/modulefiles
- Download the bootstrap installer:
svn export https://svn.sdss.org/public/repo/sdss/sdss4tools/trunk/bin/sdss4bootstrap
- Run the bootstrap installer:
./sdss4bootstrap
- Get sdss4tools into you path:
module load sdss4tools
- Steps 4 and 7 may be added to your ~/.module file, and that file sourced by your .bashrc or .tcshrc for convenience.
It will now be possible to install SDSS-IV data or software products by specifying the partial path to the product, e.g.:
sdss4install --public data/sdss/catalogs/dust v0_2
sdss4install --public sdss/idlutils v5_5_17
If you use EUPS at your institution, then sdss3tools is a more automated way to install SDSS-III archival software. The procedure below describes how to install sdss3tools, which will then allow you to install other products.
If you don't know what EUPS is, you should ignore this section.
- Install EUPS
- Download the bootstrap installer:
svn export https://svn.sdss.org/public/sdss3/repo/sdss3tools/trunk/bin/sdss3bootstrap
- Run the bootstrap installer:
/bin/bash sdss3bootstrap
- Get sdss3tools into you path:
setup sdss3tools
It will now be possible to install SDSS-III products by doing, e.g.:
sdss3install -c idlutils v5_5_14
Products in SVN (deprecated)
SDSS software products
- sdss
- apogee
- eboss
- manga
- sdss3 (archival)
Downloads
Most SDSS software should be downloaded with subversion (a.k.a. svn). Generally speaking, the products below are organized into three subdirectories.
- trunk
- holds the very latest development version of the software. Stability is not guaranteed, but if you need to track developments in 'real time', you should check out trunk.
- tags
- holds labeled versions of the software that are intended for release.
- branches
- holds active development areas that are meant to be kept separate from trunk. Only experts should look here.
If you only want a copy of the software, a subversion export is the best option. For example to download version v5_5_32 of idlutils:
svn export https://svn.sdss.org/public/repo/sdss/idlutils/tags/v5_5_32 idlutils-v5_5_32
Checking out the trunk of a product (if you really need to do this) is somewhat different:
svn co https://svn.sdss.org/public/repo/sdss/idlutils/trunk idlutils-trunk
Your checkout of the trunk can be kept up-to-date by doing 'svn up'. Consult the subversion documentation for further details.
Read-Write Access
Read-write access is limited to members of the SDSS collaboration. If you are a member of the collaboration, you can set up read-write access by following the instructions on the SDSS wiki (and you will know how to find this).
Software Docs Hosted on ReadtheDocs
- sdss-tree: Sets up a local SDSS environment
- sdss-access: Access and Download SDSS data.
- sdss-marvin: Interact with MaNGA data.
- sdss-python-template: Create new SDSS Python projects.
Requirements
Some SDSS software products require other products. Listed below are the products that require other products in order to function. In addition, there are optional software products that will enhance the functionality of some products, but will not prevent the software from operating if it is not installed.
Note that many of these products are written in and require IDL to function.
Product | Requires | Optional |
---|---|---|
idlspec2d | idlutils | elodie, platelist, specflat, speclog |
idlutils | dust, first, rass, rc3, tycho2 | |
mangacore | ||
mangadrp | idlutils,platedesign,dust,mangacore,specflat | mangapreim |
photoop | idlutils | photolog, ptcoeff, secondary_patches, speclog, yannytools |
platedesign | photoop | platelist, plate |
pydl | yannytools | idlutils |
xdqso | idlutils |